I have been commuting to work by bike for a year now and one of the challenges I have faced is that the weather changes quite dramatically from day to day in London. I had to check the weather every single morning to know exactly what type of clothes and gear to bring the office. If this sounds trivial, you have never commuted 20 km. each day in a city like London. Otherwise, you understand the combinatoric complexity of choosing between types of jacket, jerseys, under-skins, socks, gloves and all the other related gear.

Creating a twitter bot to check the weather and suggest clothes to wear is probably an over-engineered solution but it was a lot of fun and it provided me with the opportunity to play with a couple of interesting technologies in order to solve a small but meaningful end to end problem. Also, I hope this could be useful for some people.

The twitter bot will check the weather forecast for the day, just before I wake up every morning. Then, based on the temperature, wind and rain forecast at two key times: 7am and 6pm (my usual starting commuting times), it will tweet a recommendation for the clothes I should wear that day. This solution has four major components:

1. Weather forecast
2. Clothes recommendation
3. Tweet integration
4. AWS Lambda Deployment

You can find all the code in the Github repository or, of course, follow the Cycling Wardrobe bot. I do recommend to have small but meaningful personal projects for those moments when you want to explore and learn new technologies or just as a mechanism to change your focus from your day to day activities.

Weather Forecast

In order to recommend appropriate clothing and gear, we need to know the weather forecast for 7am and 6pm every day. Specifically, we are interested in the temperature, the wind and the precipitation probability. Another important piece of information would be the time for sunrise and sundown (e.g., very important in terms of high-visibility and illumination gear).

After some web searches about weather APIs, I came across OpenWeather. This service charges for hour by hour forecast but its free version covers 3-hour window predictions. and it includes temperature, wind and general atmospheric conditions (e.g., cloudy, rain, drizzle, …). This is good enough for the first iteration of the project and after a simple registration, OpenWeather provides you with an API key that is the only information you will need to interact with their API. After you have added you OpenWeather API key to a configuration file, we can get the 3-hour window forecast weather data for the next 5 days in London with the following code:

weather_api_key = config['DEFAULT']['open_weather_key']
base_url = 'http://api.openweathermap.org/data/2.5/forecast
city = 'London,uk'
request = base_url +
'?q={}&appid={}&units=metric'.format(city,
weather_api_key)

The amount of information that the API provides is richer than we are currently interested in (e.g., wind direction) and we only care about two specific moments in time, instead of every 3-hour window for the next 5 days:

def time_slot_datapoint(response, day_to_predict, timings):
for datapoint in response['list']:
dt = datetime.fromtimestamp(datapoint['dt'])
if dt.date() == day_to_predict
and dt.hour in timings:
return datapoint

def time_datapoint(response, day_to_predict, time):
# The best (3 hr) window of time to define our target
# timing will start at t-1, t or t+1
return time_slot_datapoint(response, day_to_predict,
[time-1, time, time+1])

def extract_timeslot_weather_data(datapoint):
return {'dt': datetime.fromtimestamp(datapoint['dt']),
'min_temperature': datapoint['main']['temp_min'],
'max_temperature': datapoint['main']['temp_max'],
'humidity': datapoint['main']['humidity'],
'wind': datapoint['wind']['speed'],
'meteorology': datapoint["weather"][0]
['description']}

One of the other interesting pieces of information that we will use is the time for sunrise and sunset but this information requires to call a slightly different base_url as this information is only available in today weather information (instead of forecast):

base_url = 'http://api.openweathermap.org/data/2.5/weather'
city = 'London,uk'
request = base_url +
'?q={}appid={}&units=metric'.format(city,
weather_api_key)
data = requests.get(request).json()
day_weather = {'sunrise': datetime.fromtimestamp(
data['sys']['sunrise']),
'sunset': datetime.fromtimestamp(
data['sys']['sunset'])}

 

Great, we have the first part of our twitter bot done. Now, given this weather forecast, we  have to recommend the best clothes and gear to use.

Clothes Recommendation

This the simplest part of the application but I wanted to give some context. Deciding what to wear while cycling is a more complicated subject that it sounds as there are many possible variations which is why there are a number of blog-posts about the optimal solution for different weather conditions. However, this is completely dependent on each person. In my case, I have created a first version of this decision based just on the minimum temperature of the day and have decided to create a simple if-else function in Python to return a string containing the recommendation.

min_temperature = min(weather_forecast['morning']
['min_temperature'],
weather_forecast['afternoon']
['min_temperature'])

if min_temperature < 5:
return "Wear jacket with underskin, thick gloves, " \
"long bib and thermal socks. " \
"Cover for head and neck."
elif min_temperature < 10:
return "Wear jacket, thick gloves and long bib. " \
"Cover for head and neck."
elif min_temperature < 15:
return "Wear long jersey or mallot with arm " \
"warmers, long gloves and long bib or " \
"short with leg warmers." \
"Cover for neck."
elif min_temperature < 20:
return "Wear jersey (carry arm warmers in case), " \
"fingerless gloves and short bib."
else:
return "Wear thin jersey, fingerless gloves " \
"and short bib."

This could be seen as a simplistic solution, but I believe it is enough to provide value and to test our idea end to end before spending too much time in early optimisation. Some of the aspects to improve in the future will include the rain probability, the wind intensity and to consider large differences in temperature (e.g., suggesting arm and leg warmers instead of long pieces for days with cold mornings but warm afternoons).

Tweet Recommendation

In order to have a Twitter bot, we have to of course create a new twitter account and to register a new Twitter application, as it has been described many times in other blogs. Once this is done, the amount of code to create new post, using tweepy is incredible simple as the code below shows. In our case, in addition to the message, we also added a picture representing the weather.

# Authorisation
auth = OAuthHandler(config['DEFAULT']['consumer_key'],
config['DEFAULT']['consumer_secret'])
auth.set_access_token(config['DEFAULT']['access_token'],
config['DEFAULT']['access_secret'])
api = API(auth, wait_on_rate_limit=True)

bike_emoji = u'\U0001F6B4'

# General meteorology codes from OpenWeather to
# decide the picture to show in the tweet
meteorology_codes = (weather_forecast['morning']
['meteorology_code'],
weather_forecast['afternoon']
['meteorology_code'])

# pictures for rain (codes 300-600) and clear (>800)
image = ''
if 300 <= meteorology_codes[0] < 600
or 300 = 800:
image = './images/clear.jpg'

message = '{} Weather for {} in London '
.format(bike_emoji,
weather_forecast['morning']
['dt']
.strftime("%A %d"))
message += '\nMorning: {} | {} °C. {} kph wind.'
.format(weather_forecast['morning']
['meteorology'],
weather_forecast['morning']
['min_temperature'],
weather_forecast['morning']
['wind'])
message += '\nAfternoon: {} | {} °C. {} kph. wind.\n'
.format(weather_forecast['afternoon']
['meteorology'],
weather_forecast['afternoon']
['min_temperature'],
weather_forecast['afternoon']
['wind'])

message += recommendation

if image == '':
return api.update_status(message)
else:
return api.update_with_media(image, message)

We have now completed our (very simple) bot and we are ready to connect all the parts in an end to end solution as shown in the Github repository.

AWS Lambda

At this point, we have the first version of the bot ready. It can check the weather forecast, recommend the best cycling clothes and tweet about it. However, our program will not run automatically. The last step for our bot to be complete is to deploy it in the cloud and to ensure it runs every morning and AWS Lambda is a perfect solution for this. There are many blogposts and videos about AWS Lambda and how to set the infrastructure. In our case, all we needed was to do the following:

1. Create a new function from the AWS Lambda panel
2. Modify the “handler” to be the file and function name for our main function
3. Upload the code to AWS Lambda: I did it by copying all the dependencies into the code folder (using pip freeze and pip install -r requirements.txt -t .). There are more elegant ways but this was simple enough for this exercise.
4. Create a new triggering based on CloudWatch with the pattern cron( 0 5 * * ? *). This was the code is executed every day at 5 am UTC (6am in London).

One very useful tip is to create a shell script to automate the code deployment to AWS Lambda to simplify the deployment as much as possible. In my case, I did this using the following code:

pip3 freeze > requirements.txt && \
pip3 install -r requirements.txt -t . && \
zip -r Twitter-Cycling-Bot.zip ./* && \
aws lambda update-function-code
--function-name Twitter-Cycling-Bot
--zip-file fileb://Twitter-Cycling-Bot.zip

This last step allows us to be detached from the execution and now the first version of our Twitter bot is ready.

Takeaway

This blogpost has described a small personal project that has provided me a perfect excuse to play a bit with AWS lambda and to create a simple Twitter bot while building something that might be useful for other people too. I hope this post also shows how easy it is to create a small twitter bot and how useful and simple AWS Lambda could be for this type of periodic code executions.

I will continue slowly but surely improving this project as many parts are very simple at the moment, from the clothes decision to the visualisation and style of the tweets but I am happy with the initial result and I have to say I have already been using the bot for my commute last week.