Opinion data on a platter

Joseph Allen, a Research Associate at the UKDS, discusses why we use Twitter data for research.

When we try to analyse an individual we have access to a snapshot of who this person is. We could collect quantitative data on ages, sex, family details, salary and beyond. While these data points are valuable, they only capture a part of an individuals’ complicated history and limitless potential.

Photo by Solen Feyissa from Pexels

Beyond this, we make a few assumptions:

  • Our individuals told the truth, to us and themselves. Pessimists might downplay the impact of a diet. Many of us would overstate the amount of exercise we do.
  • Events outside our dataset are…

Sensitive data, without the sensitivities.

The benefits and methods of synthetic data generation

Synthetic data is an upcoming technology in the data industry. There are many reasons to explore synthetic data, which I will go into shortly.

Sometimes the most effective way to collect more data is to make it yourself.

Green Matrix style falling text
Green Matrix style falling text
Photo by Markus Spiske from Pexels

A computer program manufactures synthetic data. We no longer measure real-world events. There are many methods for generating synthetic data. These methods can range from find and replace, all the way up to modern machine learning. The synthesis starts easy, but complexity rises with the complexity of our data. To use synthetic data you need domain knowledge. You need to understand what…

My Day 1 Setup

Set up your development environment for data

Photo by Lukas from Pexels

Python and R are the main tools in the world of data. Although they aren’t trivial to get started with, they should be. This post walks through my day 1 set up at the UK Data Service. We cover setting up any machine for Python, R and a plotting example for each. No programming knowledge needed.

The UK Data Service hosts the United Kingdom’s largest collection of social data. Through the website, you can request, manage and deposit all sorts of social data. Sharing access to that data is only the first step. …

“Sorry, I got too excited”, the recruiter blabbered.

In Manchester, we have a Community Slack which I recommend. It is a fantastic place of joyous discussion, job offers, pet projects, pets and beyond. It’s also a great place to find a job.

Photo from Pexels

One day on Slack I got a phone call from a recruiter who had sent me a message minutes before which I had not yet replied to.

“Sorry, I got too excited”

Although jarring I gave them a chance to start again but it came with no apology or change of tune. Poor representation is costing the company, good applicants. From faking family illness to contacting…

Maslow’s hierarchy of needs

How do you make a product trusted enough to convert?

Recently some friends and I have been talking about starting a company together. We still aren’t sure what that looks like and our product changes every week. We are hitting some problems around trust, and how you make a product trusted enough to convert. I can’t remember how we got here but I drew the below diagram and labelled it “The Triangle of Success” in parody.

Joe’s Triangle of Success

Since the (re)creation of “The Triangle of Success” out of a moment of boredom, I’ve noticed its power for intentional action. This is a parody of Maslow’s hierarchy of needs. To those unfamiliar, each…

Pick a topic you can discuss at length

People have asked me how to get started once I pick a topic.

I have been thinking about what blog post to write next. People have asked me how I bring myself to write a blog post. People have asked me how to get started once I pick a topic. So here it is, the meta “How to write a blog post” blog post.

First of all, you need to pick a topic you can discuss at length, something you could talk about for five minutes. This is the hard part. I am confident you already can write about something you know. My process is as follows.

Photo by Jessica Lewis from Pexels

You need a text editor. Open…

Does a generic competency test have a place in your interview?

A kata is a take-home task provided by a company.

A kata is a take-home task provided by a company as part of the interview process.

Photo by Thao Le Hoang on Unsplash

In all sorts of roles, this is a common part of the process. In Front-End it may be building a website. In Data Science it may be something like “find three insights”. In Marketing it may be “come up with a marketing strategy”, one they could steal and run away with. Whatever the kata is, it’s always free work and an expectation of best work in a small period of time. Not only is this disrespectful to the candidate’s time, but it also costs…

Make Twitter work for you.

Put the work in and Twitter will work for you.

When we own a product it’s beneficial to build trust with a community. Users often check Twitter accounts to gauge the maturity of a product. It can appear jarring to follow too many people, not tweet enough or not have a nice profile picture.

Photo by freestocks.org from Pexels

Twitter is a great platform if you want somewhere where you can repeat yourself.


At a low level, you need some followers to appear valid. The number of followers you have isn’t a useful goal. The goal isn’t to hit 1000 followers or gain 25 followers…

life is too short to give your time to a company that gives no time to you.

My experience as an intern Web Developer in Manchester

This post is a written version of a talk I give. Slides here. If you would like me to give this talk contact me here.

This is a sad story of how my first job in web development treated me, and what I learnt from it. Self-improvement and the mastery of skills trump anything a company can offer you.

The year is 2015. I am 20 years old and I love front end development. I spend my time freelancing around my degree. A break from academia sounds like a good idea so I start looking for industrial experience. …

How I hired a Data Scientist.

The industry doesn’t know how to hire Data Scientist’s, here is how I do it.

Data Science roles have unrealistic expectations from blog posts and other job descriptions. If you are going to copy one I hope you pick this one!

CV Photo by Lukas from Pexels

You are probably not an expert in Data Science. That’s okay, the fact that you are reading this puts you above other companies in the search for a Data Scientist.

What is a Data Scientist?

Data Science is an ocean as deep as it is wide. A Data Scientist is somebody who can:

  • Program.
  • Visualize and investigate data.
  • Shape a dataset and manage storage and transformation.
  • Make meaningful predictions…

Joseph Allen

Manchester based Data Scientist, Digital Artist, Front-End Dev

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store