Poll war 2023: first round

Escuchar la entrada

By Mar­cos Pérez Talia.

The polls have started to be pub­lished and with them the con­tro­ver­sies. Just as Ter­eré Cóm­plice pub­lished sev­eral ar­ti­cles about them dur­ing the last elec­toral cy­cle, we will put the mag­ni­fy­ing glass on them again.

Let’s start with the lit­tle known poll­ster: FASAC SRL. Ac­cord­ing to a re­port in the news­pa­per La Nación on Feb­ru­ary 20, it pub­lished a sur­vey that gave a 13-point ad­van­tage to Peña over Ale­gre. Among the few an­tecedents of FASAC SRL, one notes that it pro­jected a dif­fer­ence of 20.3 points in fa­vor of Peña over Wiens in the in­ter­nal elec­tions of the ANR in De­cem­ber 2022, when the dif­fer­ence was 7.7 points. In ad­di­tion, one can ob­serve in his Twit­ter ac­count tweets in fa­vor of Peña (as the im­age be­low), in­di­cat­ing his pref­er­ence and bias to­wards that can­di­dacy.

Two other lit­tle-known poll­sters that have been high­lighted by La Nación’s ar­ti­cle of Feb­ru­ary 20 are Oima Data and Mul­ti­tar­get. Oima Data has no track record and pro­jected a dif­fer­ence of more than 20 points in fa­vor of Peña against Ale­gre. Mul­ti­tar­get has a bad track record pro­ject­ing a dif­fer­ence of al­most 25 points be­tween Nene­cho and Nakayama in the 2021 mu­nic­i­pal elec­tions, when the dif­fer­ence ended up be­ing 5.4%. In the ANR in­ter­nal elec­tions they pro­jected a dif­fer­ence in fa­vor of Peña of 20 points when it ended up be­ing 7.7 points.

These three poll­sters (FASAC SRL, Oima Data and Mul­ti­tar­get) are ap­par­ently new, the 2023 gen­eral elec­tion be­ing the first in which they ap­pear pub­licly. Their re­sults are widely dis­sem­i­nated by the me­dia of Grupo La Nación, owned by Sarah Cartes, sis­ter of Ho­ra­cio Cartes.

Among the in­ex­pe­ri­enced poll­sters, with pro­jec­tions fa­vor­able to Ale­gre, is of Chioc­chini y Aso­ci­a­dos, which gives a mar­gin of 3.6 points in fa­vor of Ale­gre. This poll­ster also has no track record.

Now let’s move on to the group of poll­sters with more ex­pe­ri­ence. Among these are Ati Snead and GEO, which al­ready pub­lished polls in sev­eral gen­eral and mu­nic­i­pal elec­tions since at least 2008. In ad­di­tion, we in­clude Eco­dat, which ap­peared for the 2018 elec­tions. Ati Snead gave in Feb­ru­ary 3 more than 20 points to Peña, Eco­dat on Feb­ru­ary 20 10 points to Peña and GEO also on Feb­ru­ary 20 gives 3.7 points to Ale­gre. New Ati and GEO polls have al­ready come out af­ter these which we will an­a­lyze in a fol­low­ing ar­ti­cle.

In this first round of surveys, GEO stands out with, at least, a better sample design.

Eco­dat is the least ex­pe­ri­enced among the three and with a his­tory of sig­nif­i­cant er­ror in its pro­jec­tion. In 2018 it pro­jected a dif­fer­ence be­tween Abdo and Ale­gre of about 25 points, when the ac­tual dif­fer­ence was 3.7 points.

GEO and Ati Snead are the most ex­pe­ri­enced. In the 2008 elec­tions both pro­jected Lu­go’s vic­tory. In the 2013 elec­tions both Ati and Geo pro­jected a nar­row mar­gin vic­tory for Ale­gre over Cartes, a re­sult far short of Cartes’ 8.9% vic­tory. In the 2015 mu­nic­i­pal elec­tions Ati gave an 18-point ad­van­tage of Fer­reiro over Samaniego in Asun­ción, while GEO gave al­most 11 points. The dif­fer­ence was 10.5 points.

In these com­par­isons, GEO and Ati were quite sim­i­lar, with GEO hav­ing a bet­ter ap­prox­i­ma­tion in 2008 and 2015.

Ati then was the clos­est in 2018, but in the 2021 mu­nic­i­pal elec­tions their poll gave 20 points in fa­vor of Nene­cho and the exit poll gave a dif­fer­ence of 13 points, when the fi­nal dif­fer­ence was only 5.4 points. In the ANR in­ter­nal elec­tions, his last poll pro­jected a dif­fer­ence in fa­vor of Peña of 16.4 points when the dif­fer­ence was 7.7%.

In con­clu­sion, Eco­dat has lit­tle ex­pe­ri­ence with a very large er­ror in the 2018 elec­tions. Ati and GEO have a lot of ex­pe­ri­ence, with mixed re­sults, al­though GEO has bet­ter ac­cu­racy. While Ati had a good pro­jec­tion in 2018, both in the 2021 mu­nic­i­pal elec­tions and in the 2022 NRA in­ter­nal elec­tions, its pro­jec­tions were quite far from the re­sults.

Let us now turn to an­other im­por­tant point for analy­sis. Be­yond his­tory, we can an­a­lyze the “sam­ple de­sign” used by the poll­sters, pro­vided they have pub­lished the de­tails.

The sam­ple se­lected by each poll­ster should have cer­tain char­ac­ter­is­tics con­sis­tent with voter reg­is­tra­tion and his­tor­i­cal voter turnout data. For ex­am­ple, half of the poll­sters should be male, since male vot­ers tend to be about half. Also, Asun­ción and Cen­tral should be about 40% of the re­spon­dents, since that is the pro­por­tion in the voter list. Cer­tain pro­por­tions should also be re­spected by age, po­lit­i­cal party pref­er­ence, in­come range, ur­ban/​rural area, among oth­ers.

In the case of the Eco­dat, Ati and GEO sur­veys we can an­a­lyze the sam­ple by age range and com­pare with what the turnout was in the 2018 gen­eral elec­tion. GEO di­rectly showed the per­cent­ages of their sam­ple by age range.  From the Ati and Eco­dat polls we can in­fer the num­ber of re­spon­dents in each age range, given that we have the pub­lished pref­er­ences by age range.

In the fol­low­ing fig­ures we can see what the weight by age range of the 2018 elec­tion turnout and polls was:

Graph 1: Weights by age range. ECO­DAT

Source: own cal­cu­la­tions based on TSJE and ECO­DAT.

 

Graph 2: Weights by age range. Ati Snead

Source: own cre­ation based on TSJE and Ati Snead.

Graph 3: Weights by age range. GEO

Source: own cre­ation based on TSJE and GEO.

Of the three, ECO­DAT is the poll­ster with the worst sam­ple. Not sur­pris­ingly, it had a mar­gin of er­ror of about 20 points in the 2018 elec­tion. The Ati poll has a lot of young peo­ple and few over 60, rel­a­tive to what hap­pened in 2018. The GEO poll, on the other hand, is close to what it should be.

If a sur­vey takes a bad sam­ple, its re­sults are not rep­re­sen­ta­tive, and this is key. They are sim­ply use­less. It is the good choice of sam­ple that al­lows a poll of about 1500 peo­ple to be rep­re­sen­ta­tive of mil­lions of votes.

There­fore, the Eco­dat sur­vey should be dis­carded as in­con­sis­tent. The Ati sur­vey has se­ri­ous prob­lems that should be cor­rected in fu­ture pro­jec­tions. The GEO sur­vey has a rep­re­sen­ta­tive sam­ple (at least in age range), so it seems to be well de­signed.

In this first round of sur­veys, GEO stands out with, at least, a bet­ter sam­ple de­sign.

Cover im­age: cam­bio24.com.co