ELXN43 and #CdnPoli: Exploring Sentiment on Political Twitter


File it under “Fun Things To Do During An Election.” We’re going to track the sentiment scores of tweets on the ‘official’ Canadian politics hashtag on Twitter, #CdnPoli, first and foremost just to gather the data but also to see if we can learn anything about the political process on Twitter. There are any number of research questions that one can use to approach this data set but for right now we’re just going to use what I refer to only half-jokingly as the Armstrong Method: start with the data and then explore it to see if any patterns leap out at you, before you start making assumptions about it. It’s a much more inductive process and one that this type of data set seems to really need.

So how does this work? First we scrape #CdnPoli, of course. I have a standard script I like to use to do that using the twitteR package in R; I’m aware that RTweet is newer, better (and, more importantly, maintained) and I use that as well, but for some reason I can’t get RTweet scripts to schedule properly using Windows Task Scheduler so for right now the workflow is: scrape tweets initially using twitteR, and then run them through an iterated loop that re-grabs each tweet in RTweet. This is not ideal obviously and takes slightly longer than if I could just get RTweet to schedule properly, but it does have the added bonus of eliminating duplicate tweets without having to take an extra step, so maybe it washes out.

Tweets get scraped on an hourly basis and then stitched together at the end of the day into one csv file. The Hu & Liu sentiment lexicon is used to provide scores for sentiment in each tweet. The Hu & Liu lexicon is a dictionary of around 6800 English words that are coded to denote positive or negative sentiment. If a tweet contains a word in the lexicon, the sentiment score of the tweet is adjusted up or down as necessary. This is a fairly basic application of sentiment analysis but it does allow for some numerical representation of the sentiment of a tweet.

Next, regular expressions are used to identify tweets referring to each of the five ‘parties of interest’ in the 2019 Canadian federal election and to each of the leaders of those parties. Separating each of these out allows us to determine a mean score for tweets mentioning that party or party leader. We can then track that sentiment day-by-day (I mean, we could do it hour by hour but come on, I have grad work I’m supposed to be doing).

Just as a visual example of what this is, here’s a bar chart for Day 1 (September 11th, 2019):


So, on the first day of the campaign, tweets using the #CndPoli hashtag were quite negative with regard to both the Liberal Party and Prime Minister Justin Trudeau; they were also negative (although not quite as negative) with regard to the Conservative Party and CPC leader Andrew Scheer. Positive opinions were characteristic of tweets regarding the New Democratic Party, the Green Party, the People’s Party, and their respective leaders. Overall, the sentiment on #CdnPoli for Day 1 was slightly negative.

One of the possibilities for the sentiment scores we’ll see over the election is that they may be a measure for how energetic the respective campaign is. About 90% of Canada is online, and 61% of Canadians use social media on a daily basis. 77% of Canadians use Facebook, whereas 26% use Twitter. Despite this, Twitter makes for a more interesting data source for two reasons: first, Twitter posts are publicly available and Facebook posts are jealously guarded by Facebook; second, natural limitations on Twitter posting make for a more level playing field with which to analyze posts. Posts cannot be longer than 280 characters, which eliminates the scaling problem that typically occurs when you have texts of wildly differing lengths to analyze. Everyone’s posts are around the same length, so we can compare one tweet to the next in terms of sentiment scores derived from them. The use of hashtags as organizing containers on Twitter makes it easy to track specific phenomena of interest, which means that data gathering can be accomplished with relative ease.

The rationale for sentiment scores being a function of campaign energy lies in the relatively low use of Twitter by Canadians. If only 26% of Canadians are using Twitter, then the ones who are both using the platform and are specifically engaging on an explicitly political hashtag are likely to be more engaged politically than the average person. They are more likely to be partisan supporters of a specific party (or at least a range of parties within a certain ideological range). Thus, we can think of it as persons performing one of two activities: boosting support for their chosen party by expressing positive sentiment, or attacking an opposing party by expressing negative sentiment. If supporters can manage to boost their chosen party more than opposers can manage to bring down their opposing party, then sentiment will rise day-over-day. This represents an energized campaign. If the opposite occurs, and supporters can not express enough positive sentiment to overcome the negative sentiment of their opposite numbers, then sentiment will fall day-over-day. This represents a de-energized campaign.

Granted, there are also users of #CdnPoli who are not hard supporters of one party or another; these can be thought of as ‘undecided’ voters reacting to acute or aggregate events during the campaign. These will also effect sentiment scores on a daily basis; if a particular event causes negative sentiment outside of social media toward a party, then their supporters will have a harder time boosting the party through positive sentiment tweets.

At least, that’s a theory. A proper academic treatment of the subject after the fact will require some grounding in the literature obviously but that’s at least something to hang our hats on with regard to tracking it during the campaign. In the end, we’ll see what the data says.

As an example of tracking change day-over-day, here’s each party by day for the first three days of the campaign:


and the same, but for party leaders:


Note the upswing for May and Singh and their respective parties after day 2; the first debate occurred on that day and we can see a sharp effect for them and a slight upswing for Scheer. These three leaders participated in the debate, whereas Trudeau chose not to take part (Bernier was not invited). Despite this, we see a downward trend on that day for Scheer’s Conservative Party and a sharp upward trend for the ruling Liberal Party. Part of this likely has to do with perceptions gained from the debate; I’ll post a bar chart of a separate analysis of just the #FirstDebate hashtag later, but the general idea is that Jagmeet Singh won the sentiment battle there, while Scheer underperformed and in fact garnered less positive sentiment than Prime Minister Trudeau, who wasn’t even there.

At any rate, that’s the idea. Let’s see what the data tells us from here on out.

Consumer Guides?


They’re a lot of work when you’ve got a thousand other commitments, so I might switch to doing monthly, or at least bi-weekly? The text mining stuff gets repeated reads and is more in line with skills I’m developing to exchange for money, so I would really prefer to do more of them, but I also love music and this is the last year of the 2010s and so there are retrospectives and context to think of for 2019. I just haven’t been enthused this year, beyond a few key records (Sharon Van Etten I’m looking at you), and it’s hard to muster up the get-go to, you know, plow through it. So, be on the look out. Or don’t. I’m not a cop.

Consumer Guide March 8th/2019


Sigrid – Sucker Punch


(March 8th on Island Records)

Europe has always had pop figured out cold, Northern Europe doubly so. Norwegian singer Sigrid’s debut follows in this tradition, trading in solid pop sensibilities that don’t chase any particular trend or try to reinvent the wheel. It plays it safe, but that safety is also so well done it feels like it might be something more. It’s also distressingly wholesome, but the message – good things come to those who love themselves – needs to be heard more.

Foals – Everything Not Saved Will Be Lost, Part 1


(March 8th on Warner Bros. Records)

Literally the Imagine Dragons of indie rock, if you can stomach such a thing. I never have been able to, but there are stronger constitutions out there than mine, I’m sure.

Sasami – Sasami


(March 8th on Domino Recording Company)

An interesting debut from the one-time Cherry Glazerr member, this one is quiet and intimate while managing to to call forth some real heavy power when it needs to.

Amanda Palmer – There Will Be No Intermission


(March 8th on Cooking Vinyl Records)

The stark nudity of the record cover is a metaphor, for the ultra-confessional proceedings offered here by the Dresden Dolls singer. Imagine if Mark Kozelek had something useful to say, and it might sound something like this.

Sundara Karma – Ulfila’s Alphabet


(March 8th on RCA Records)

Combining Wolf Parade with Eighties Bowie is interesting, and…that’s pretty much where it ends.

Meat Puppets – Dusty Notes


(March 8th on Megaforce Records)

Their version of country-punk was innovative thirty plus years ago but in this day and age when country is grudgingly allowing again for dissonant voices it rings a little hollow.

William Basinski – On Time Out Of Time


(March 8th on Temporary Residence Records)

It sounds like two supermassive black holes colliding 1.3 billion years ago.


Because that’s what it is.

Literal, inescapable doom.

Townes Van Zandt – Sky Blue


(March 7th on Fat Possum Records)

A collection of demos from the 22-year-dead balladeer of the dusty forgotten highways and the decaying dive bars, barely more than shacks themselves. Sparse, desperate, doomed American music.

Dido – Still On My Mind


(March 8th on BMG Records)

Pretty, too polished, but pretty. I actually like it more than her big hit album from 2000 or whenever, the one everyone knows because of “Stan”.

Stella Donnelly – Beware Of The Dogs


(March 8th on Secretly Canadian Records)

Frank, blunt, and sexual, like Lily Allen emerging sharper than ever from the #MeToo era. It will make some people uncomfortable. Be extremely wary of the people it makes uncomfortable. Four stars only because it kind of meanders after the highwater mark of “Boys Will Be Boys” before coming in strong again at the end.

Maren Morris – Girl


(March 8th on SME Records)

Excuse me, wasn’t this supposed to be a country album?

Helado Negro – This Is How You Smile


(March 8th on RVNG International)

Weirdly exuberant for such an insular, spacey record. This is how you know they found that good stuff.

Nick Waterhouse – Nick Waterhouse


(March 8th on Innovative Leisure Records)

Devoted to a certain time and place – America, 1963 – but maybe a little too on the nose for much of it’s run.



Consumer Guide, March 1st/2019


Royal Trux – White Stuff


(March 1st on Fat Possum Records)

When a band comes back after twenty years to release a new album I expect them to play it safe but there’s such a thing as taking it too far. White Stuff is fine, especially if you were into the band Back In The Day, but it sounds like a rewrite of older, better stuff with no attempt at trying to move forward at all.

Continue reading

Consumer Guide, February 22/2019


Teeth Of The Sea – Wraith


(February 22nd on Rocket Recordings)

Like a soundtrack for apocalyptic times, Wraith veers between pounding industrial, jazzy, introspective diversions, and moodier alt-psych excursions. Like the zee, it changes when you don’t look at it for too long.

Continue reading

Consumer Guide, February 15/2019


Woman’s Hour – Ephyra


They broke up after their acclaimed 2014 debut but managed to stick together long enough to put together this enchanting, disturbing synth-pop confection. Nineties Bjork and Oughts-era The Knife with a gauzy layer that crinkles uncomfortably like human skin when you press it slightly.

Continue reading

Consumer Guide, February 8th/2019


Cass McCombs – Tip Of The Sphere


(February 8th on Anti-Records and Epitaph Records)

Cass McCombs is great because his heady folk rock gets Americana and jammy but just before that gets three-Phish-sets-at-Bonnaroo obnoxious he veers off into another territory entirely. One day we’ll probably think of him as a Great Lost American Songwriter but you could just discover him now, too, it’s never too late.

Continue reading

Consumer Guide: February 1st/2019


Hedvig Mollestad Trio – Smells Funny


(February 1st on Rune Grammofon Records)

The worst thing about fusion has always been how seriously it’s practitioners take it. It’s basically all the power of both jazz and rock ‘n’ roll put into a tuxedo and made to look for respectability among the moneyed classes of the world. The Hedvig Mollestad Trio, however, wants no such respectability. Their fusion is haunted by the ghost of Jimi Hendrix, and as such it’s explosive, heavy, and outfitted with guitar lines that whip and strangle as much as they slither and caress.

Continue reading

Consumer Guide: January 25/2018


TOY – Happy In The Hollow


(January 25th Tough Love Records)

Low-tempo Krautrock with a seriously languid groove. If Faust was secretly a bunch of goddamn hippies they would sound like this.

Continue reading

Consumer Guide: January 18/2019


Lorelle Meets The Obsolete – De Facto


(January 11th on Sonic Cathedral Records)

Psychedelic noise-rock that walks the thin line between being artistically and willfully difficult. Avoids becoming lost in a gauzy haze by virtue of an excellent rhythm section that knows instinctively how to ride a groove.

Continue reading