A published poll is the end product of decisions most readers never see: who could be sampled, how they were contacted, when the interviews took place and how the sample was adjusted to resemble the population. In Australia and New Zealand, each of those decisions has changed over the past decade. The landline frame that once supported most telephone polling now reaches a smaller and older share of voters, and pollsters have moved to mobile numbers, online panels and combinations of the two.
This note describes how political polls in both countries are conducted, what the industry codes require pollsters to disclose, and why electorate, ward and council polls remain harder than national ones. It ends with where AI telephone interviewing fits.
From landline samples to mobiles and panels
Randomly generated landline numbers were long the backbone of computer-assisted telephone interviewing for polls. According to Verian’s FAQ for the 1News Verian poll, that New Zealand poll was conducted entirely by calling randomly generated landline numbers until 2017. Declining landline access and use made representative samples harder to collect, so it moved to mobile and landline calling in 2018, then removed landlines after further testing at the 2020 election. Half of each sample of around 1,000 eligible voters now comes from calls to randomly generated mobile numbers, and half from people who have signed up for online market research.
Australian telephone surveys followed a similar path, with dual-frame designs combining landline and mobile numbers becoming the predominant approach. A 2020 methodology conference presentation on sub-national telephone surveys noted that landlines could reach only 49% of the population, with coverage skewed towards older age groups. Published polling moved further. Writing after the 2019 federal election, the head of the industry’s inquiry panel pointed to the “increasing take-up of lower cost polling methodologies (predominately online and robopolling)”, the automated telephone polls discussed in our note on IVR surveys. Mixed designs combine modes with different coverage and measurement properties, as our comparison of CATI, CAWI and CAPI explains.
The 2019 miss and the inquiry that followed
At the 2019 Australian federal election, the final polls converged unusually closely and were wrong in the same direction. An inquiry run by the Association of Market and Social Research Organisations (AMSRO), with the Statistical Society of Australia closely involved and a panel led by Darren Pennay, released its final report in November 2020. According to the Statistical Society’s summary, the errors lay in the polls themselves rather than in a last-minute shift among voters, and arose from “inadequately adjusted, unrepresentative samples” that over-represented more politically engaged and better-educated voters.
Two findings matter for anyone reading a poll today. First, as Pennay wrote afterwards, that skew was not reduced by the sample balancing or weighting strategies pollsters used; weighting by educational attainment looked promising. Second, the panel said a lack of access to data sets and detailed methods limited its ability to identify every contributing factor, which is precisely the information disclosure is meant to provide. It also found that reporting of the polls had not met the Australian Press Council’s basic disclosure guidelines. Among its 10 key recommendations was an enforceable code of conduct for election polling.
What the Australian Polling Council requires
Pollsters also launched the Australian Polling Council. Its code of conduct, in a version dated June 2021, binds members whenever publicly released polling concerns political issues, government performance or public policy, not only voting intention. Each published poll needs a short methodology statement giving the research company, the client, fieldwork dates, target population, sample size and mode. Automated telephone polls must be clearly distinguished from operator-administered ones, and a mixed-mode poll must give the share of completes from each mode.
A long statement adds the effective sample size or weighting efficiency, the margin of error for that effective sample, the weighting variables and the source of their targets, the weighting method, full question wording and any earlier questions that may have influenced the published ones. Telephone polls give the proportion of interviews on mobiles and landlines; online polls say whether respondents came from a panel, river sampling or an opt-in survey. Voting-intention polls should ask that question before anything that could influence it, state how undecided respondents were treated, and say whether a two-party-preferred figure comes from stated preferences or historic preference flows.
New Zealand’s political polling code
New Zealand’s equivalent is the New Zealand Political Polling Code of the Research Association New Zealand (RANZ). First published in 2014 and updated in 2020, it binds RANZ member companies and researchers and covers polls on public votes, including national elections, local body elections and referenda.
The poll report must give the sampling method, sample size, maximum margin of error, collection dates and the share undecided or refusing. It must state whether landlines, mobiles or both were used, whether any online panel complies with ESOMAR’s guideline for online research, and whether robo-polling was used, and it must be published on the agency’s website. The code also recommends reporting the median date of collection and the spread of weights. Because voting in New Zealand is not compulsory, unlike Australia, the code asks reports to define how likely voters were identified. Nationwide polls must have at least 500 respondents; regional and electorate-wide polls at least 250.
Reading a methods statement
- Mode and sample source. Randomly generated mobile numbers, a listed sample and an opt-in panel reach different people by different routes. The mode shares show how much each route contributes.
- Field dates. A poll measures opinion during its field period, not on the day it is published. A period that straddles a major event deserves a closer look.
- Weighting. The variables show which imbalances were corrected; the effective sample size shows what the correction cost in precision. As 2019 showed, weighting on standard demographics cannot repair a skew it does not measure, a point developed in our note on response rates and the limits of adjustment.
- Question order and wording. Voting intention asked after questions on a contested issue is a different measurement from voting intention asked first.
Why electorate, ward and council polls are harder
Electorate polling and council polls face every problem a national poll faces, with fewer people to draw on. The New Zealand code’s minimum of 250 for an electorate-wide poll implies a maximum sampling margin of error of about ±6.2 percentage points at 95% confidence for a simple random sample, before weighting widens it.
The harder problem is reaching the right people. The 2020 conference presentation cited above put it directly: “Mobile telephones are not inherently linked to a geographic location.” It described geographically targeted mobile samples as expensive, and noted that listed mobile numbers with location attached have coverage errors, particularly for younger age groups. When the Social Research Centre ran a dual-frame survey for the Australian Bureau of Statistics in 2015, no geographic information was available for mobile numbers generated from known prefixes, so the mobile sample was drawn as a single stratum. Online panels raise a parallel problem: a national panel may hold few members in any one ward.
A small-area poll drawn from a broad mobile frame therefore has to establish where each respondent lives, usually by asking for a suburb or locality and mapping it to the boundary rather than relying on people to name their electorate or ward. Many calls end at that screener, and quotas by age, gender and area fill unevenly. The methods statement should say how area was established.
Where AI telephone interviewing fits
AI polling here means telephone interviews conducted by an AI voice interviewer rather than a person or a recorded script. It changes fieldwork capacity, not the frame. Because interviews run in parallel, interviewer seats no longer limit how many calls are in progress, so a poll of randomly generated mobile numbers can close in a field window nearer to what online panels offer, within permitted calling hours. Every respondent hears the same wording in the same order, which makes a rule such as asking voting intention first simple to keep. Quotas can be screened during the interview, so in a small-area poll a respondent whose cell is already full can be thanked at the screener and effort directed to lagging cells.
Coverage, nonresponse and weighting still decide accuracy; a poll fielded quickly from an unrepresentative sample is still unrepresentative. Disclosure matters too. An AI-interviewed poll is not operator-administered, and the Australian code requires automated telephone polls to be clearly distinguished. New Zealand’s code describes robo-polling as involving auto-diallers and automated voice interviews, says it “has been found to be less accurate than live interviewer polls due to non-response bias and self-selection by participants”, and requires reports to say when it was used.
That judgement appears in the code’s 2020 version, which predates conversational AI interviewers, and the problems it names are the ones they are designed to address. A recorded robo-poll cannot persuade, explain or keep a respondent on the line, which is why so many people hang up partway; our note on IVR surveys shows how large that gap can be. A conversational interviewer can do all three. Whether it closes the gap with live interviewing is an empirical question, so test it against live-interviewer fieldwork before relying on it for a published series, as our note on evaluating AI voice interviewers describes, and name the mode plainly in the methods statement.
Research-call rules in Australia
Polling calls are research calls under the Telecommunications (Telemarketing and Research Calls) Industry Standard 2017. They are permitted between 9 am and 8.30 pm on weekdays and between 9 am and 5 pm on weekends, and not at all on specified national public holidays, with times measured at the account-holder’s usual residential address. A mobile number does not reveal that address, and the standard’s own example concerns a mobile in a different time zone, so calling windows for a national mobile sample need to allow for time zones.
At the start of a call, the caller must state its purpose and, in most cases, the calling organisation’s name. A given name is not required when the call uses only a recorded or synthetic voice, but such calls must offer a way to request contact details for the caller, the commissioning organisation and complaints. Calling line identification must be enabled, and the call must end if the person asks.
The ACMA states that market and social researchers can call numbers on the Do Not Call Register to conduct opinion polling and standard questionnaire-based research, but that a call with any commercial-type purpose counts as telemarketing and cannot be made to a registered number. The industry standard applies either way. For New Zealand fieldwork, confirm calling practice against the RANZ codes and current law before the first call.
catilyst conducts telephone interviews with AI voice interviewers in parallel, following the researcher’s questionnaire, with quotas screened during the interview, calling windows and retry rules, a live view of completions, and structured responses exported as CSV. Decisions about the frame, weighting and disclosure stay with the researcher. See how it works, or read about national, state and council polling.
Planning a national, state or council poll?
Tell the Voxworks team the geography, sample size and field window you have in mind to discuss sample sources, quotas and fielding the poll with catilyst.