After 11 years on the App store, 12 versions released, 5k downloads, countless Tweets made through it, my GAA Scoreboard app is no more.
I got a number of e-mails last week reporting it wasn’t working anymore and when I checked the Twitter Developer dashboard, there was a message saying that the Twitter app I’d set up had been suspended. The message simply said:
This App has violated Twitter Rules and policies. As a result, it can no longer be accessed. For assistance, submit a support ticket.
No details on what rule and/or policy had been violated so I had a look at how to submit a support ticket…and there doesn’t actually seem to be a way to do that (for free accounts at least).
My best guess is that the API it is using is no longer supported properly and I’d need to update to the v2 API. Given the small number of downloads the app has had over the last couple of years, I’ve decided to retire it rather than spend any more time on it. There’s a few alternatives out there anyway so I don’t think my app will be missed.
Still, it’s somewhat annoying there isn’t a way to sort this easily given it was adding value to Twitter. C’est la vie under the new management.
Myself and my friends have been playing a Pathfinder campaign as a distraction from all the Covid stuff for the last year. To give our regular DM a break and a chance to play himself, I’ve been running some side adventures at appropriate times in the main campaign.
To try and have something different from the undead heavy I went hunting for something with some classic dungeon creatures – namely Mimics and Rust Monsters. I couldn’t find an existing adventure that fit my needs and reading other people’s adventures gave me the itch to try my hand at my own.
Given it’s been a long time since I wrote anything like this (probably 25+ years!), it’s possibly a little uneven on the challenge rating of the encounters but most should be adjustable up and down with the addition/removal of a similar creature. We did this with a group of six 5th level characters.
The maps were created using Inkarnate. There are links at the bottom to a zip containing versions usable in Roll20 as well as to editable versions on Inkarnate.
All feedback on it is welcome and I’d love to hear from anyone that actually runs it.
Background
An Intellect devourer has set up a lab in an abandoned crystal mine to try and create a Rust Dragon to help defeat his rival who has gone down the road of clockworks to build an army.
To this end, he has trapped four Rust Monsters and is force feeding three of them metals and channelling energy from them into the fourth to force a transformation into a Rust Dragon. The fourth is in a metal chrysalis and is near maturation. This is only the latest in a line of failed experiments but each attempt is bringing him closer to success.
To feed the Rust Monsters, he has tasked a band of low grade mercenaries to raid ore caravans. As the experiments have increased in pace, so has the need for ore and thus an increase in raids.
The local authorities (be it a lord, mayor or a group of annoyed mine owners) are sick of the raids so hire the party to put an end to them.
In parallel, his rival has caught wind of the plan and sent a squad of his Clockwork Soldiers to put a stop to the experiment. They slaughtered the bandits and made their way to 2.7 The Hatchery and did some damage to the chrysalis but were stopped from destroying it.
The party will coincidentally arrive at the camp within hours of this happening.
In general, there is little treasure until the end – anything with significant amounts of metal tends to end up in the hatchery and non-metallic magic items made their way to the intellect devourer’s horde.
Intro
Bandits have been raiding caravans over the last few months. After the last raid, we caught a break and one of our rangers was able to track them back to their base of operations at the head of an abandoned mine. He was unfortunately spotted and wounded while escaping. Deal with them however you want and you’ll be rewarded with XYZ.
Can the ranger come
No, injured and recovering.
What’s the make up of the bandits?
About 10, humans, nothing notable about them. Standard looking gear.
What have they been targeting?
Tending to go for ore shipments from the local mines.
1. The Camp
As the party approaches the camp, the party sees evidence of a recent skirmish. On entering the camp, they see the bodies of slaughtered bandits.
Past the barricades, the bandit camp sits in a hollow surrounded on 3 sides by cliffs. To the north of the camp is a filthy pool with steam slowing rolling off it. To the east is a cluster of tents around a smoking fire pit.
If the party tries to find clues (e.g through tracking), they notice that the foot prints of the attackers are deeper than you’d expect.
1.1 The pool
Steam continuously rolls of the pool. the water is choked with scum and algae. The edges are discoloured and rotten. A stream flows from the cliff. A broken gate is in front of the outlet.
This is the runoff from area 2.3 Drain. The party can successfully navigate the drain with a DC 15 Survival check.
If the party investigate the pool, they must pass a DC 15 perception check to spot the gelatinous orbs.
CR 6 – 4 orbs, 2,400XP
CR 7 – 5 orbs, 3,000 XP
1.2 The tents
There’s not much of value in the tents – some spare clothes and equipment belonging to the bandits.
1.3 The bodies
One of the bandits is barely clinging on to life. The party can stabilise them enough for some quick questions but he’s too far gone to save (healing potions/spells will barely prolong his life). The party can ask him a few questions before he passes away.
What the party can learn:
5 or 6 attackers – large armoured humanoids – too big to be human but too small to be ogres
Relentless and brutal – killed everyone that went against them
Armed with great clubs, mauls and obsidian swords
Weapons seemed to bounce off them
Likely after the boss’ “special project” (will die after revealing this)
2 The Mine
The tunnels are between 10 and 15 ft wide and high. There are crystals in the walls that glow faintly meaning the complex is dimly lit so the party doesn’t necessarily need their own light sources (see Vision and Light). The mine track looks well maintained.
2.1 Store room
This area is stocked high with miscellaneous goods – all obviously taken from the raided trade caravans.
If a party member makes a DC 20 perception check they notice that there are unusually few metallic objects in the store room.
2.2 Cave-in
The tunnel suddenly ends in a cave-in, covering the rail tracks and completely blocking the way forward.
A successful DC 13 Knowledge (Dungeoneering) or Knowledge: (Engineering) check will tell the players that this cave-in was deliberate and that it would take hours to clear.
If the party tries to clear the rubble, the Damaged Clockwork Solider from 2.4 Collapsed Bridge will attack them from behind.
2.3 Drain
A cave-in blocks the entrance to the small cave. Tracks run from other the rubble and end at a wall. At the top of the wall, a platform and crane are visible. A waste pipe dribbles slime and water into a small stream that flows from under the cliff and into a small pool.
The drain connects to the outlet that feeds 1.1 Pool.
If the party didn’t encounter the the Gelatinous Orbs at 1.1 Pool, they are lurking in the stream, waiting to pounce.
If the party didn’t encounter the Clockwork Soldiers in 2.10 The Lab, they are here ransacking the room. They are armed with +1 Obsidian Halberds but are otherwise normal Clockwork Soldiers.
2.4 Collapsed Bridge
The ruins of a bridge are clearly evident in the water it once spanned. An armoured body lies sprawled on an island in the middle of the pool.
It takes a DC 17 Acrobatics check to jump across the water and land on the island.
A badly battered armoured figure lies facedown on the ground. Oil rather than blood leaks from the cracks in the armour. One of the arms has been torn off and shards of obsidian from a broken halberd surround the other.
A successful DC 17 perception check and the players will see that the solider is still active. It is at ~50% health and will use the element of surprise to try and get the drop on the player examining them. They only have a single punch attack.
If the players search for the missing arm, it is at the bottom of the pool and can be retrieved with a DC 13 Swim check but there’s only the one gauntlet .
2.5 The Fungus Cave
As you enter the tunnel, there is a faint sparkle in the air. The further you go down the heavier the air gets with spores. Five huge mushrooms loom over a windlass. A chain runs from the windlass into the cavern wall.
A trio of Violet Fungus and pair of Shriekers guard the windlass and will attack anyone who approach.
CR 7, 3,200XP
On examining with windlass, the party finds it damaged. If they can repair it, winding it lifts the stone gate in 2.6 The Stone Gate and rotates the turntable. They can repair with a DC 15 Disable Device check. Using supplies from 2.1 Store Room gives a +2 bonus to the repair efforts.
Using area effect spells used to kill the fungus would further damage the mechanism making it more difficult to repair.
2.6 The Stone Gate
The rail ends in a large turntable with the rail rotated 90 degrees. A large stone blocks the way forward. A chain runs from the top of the stone into the ceiling. Crushed beneath the door is another armoured figure. Oil is smeared across the turntable – it obviously rotated after the clockwork solider was trapped.
The windlass in 2.5 The Fungus Cave will lift the door and rotate the turntable.
2.7 The Hatchery
The cavern is lit by three crystal pillars that rise out of the floor. Suspended between them is a giant, metallic chrysalis. You can see something squirming in it and small tears are visible in the shell.
Around the pillars are drifts of metallic looking powder with partially rusted items scattered on top. As the pillars pulse, the remaining items visibly decay.
What appear to be shadows dancing in the interior of the pillars resolve themselves into large, lobster like creatures with long feathery antenna and a clubbed tail. They’re obviously in agony and look like they are screaming in pain every time the pillars pulse.
On the far side of the cavern there is a wooden ladder leading up to an overhang.
The cavern ceiling is between 50 and 60ft high and covered in stalactites. The cavern floor is clear between the entrance and around the pillars. Moving through the stalagmites counts as difficult terrain.
The pillars pulse every 30 seconds (5 rounds). If a player is within 5ft of an intact pillar when it pulses, each metallic item they are wearing/wielding (be reasonable here – don’t roll on each coin!) they must make a save as per the Rust ability of a Rust Monster:
Rust (Su)
A rust monster‘s antennae are a primary touch attack that causes any metal object they touch to swiftly rust and corrode. The object touched takes half its maximum hp in damage and gains the broken condition—a second hit destroys the item. A rust monster never provokes attacks of opportunity by attempting to strike a weapon with its antennae. Against creatures made of metal, a rust monster‘s antennae deal 3d6+5 points of damage. An attended object, any magic object, or a metal creature can attempt a DC 15 Reflex save to negate this effect. The save DC is Constitution-based.
The Hatchery is protected by a Cloaker and a trio of Darkmantles. The Cloaker has cultivated a colony of Piercers (12 CR 1/4, 3 CR1 and 1 CR2) that lurk in the ceiling.
Trapped in the crystal pillars are Rust Monsters which will be released when the chrysalis cracks open. The chrysalis contains a fourth Rust Monster that is being force grown into a Rust Dragon. However, this transformation has been a failure and the creature that emerges will die rapidly in a short, agonising frenzy. The party will arrive just before the creature will attempt to break free. Any interference with the chrysalis will only speed this process up.
If the party investigate the three armoured bodies, they will see that their armour is severely dented with large holes that wouldn’t be normal weapons damage. While they are lingering over the bodies, the Piecers start to gather to drop. If the party is suspicious, a DC 20 Perception or DC 16 Dungeoneering check will let them spot the Piecers before they drop.
After the Piecers have dropped, the Cloaker and Darkmantles will take advantage of the confusion and attack. The Darkmantles will cast darkness on themselves and try and keep the Cloaker in the intersection of the radius so it is in complete darkness and as a secondary priority try to keep the party in darkness.
As the party finishes fighting the Cloaker and his cronies, the chrysalis reaches maturation.
The chrysalis splits open as the creature contained within struggles to free itself. It is badly deformed – it is like a cross between a dragon and insect with large feathered feelers sprouting from the head. Tumours run up and down its body and some organs have grown on the outside. It lets out a roar before collapsing onto the ground and pulling the pillars down with it. With its final pained breath, it spews a torrent of vile reddish-brown liquid.
Any players in a 30ft cone need to make a DC 15 Reflex save. If the fail, they then need to save for their worn/wielded items as if hit by the Rust attack described above.
The broken pillars will no longer pulse when broken, but the Rust Monsters contained within are released and will attack the nearest creatures in a fit of pain-driven rage.
There are three +1 Obsidian Halberds lying beside the remains of the Clockwork Soldiers.
The ladder on the far side of the cavern leads to a trapdoor barred with Arcane Lock.
Overall this is a CR 10 encounter but is mitigated by the fact that there are three distinct phases. The party should barely be able to get their breath back between them and pressure should be maintained through out.
Piecers: CR 7, 3,000 XP
Cloaker and Darkmantles: CR 6, 2,800 XP
Rust Monsters: CR 6, 2,400 XP
2.8 The Observatory
A large desk sits in front of a crystal window overlooking a cavern It is flanked by an orrery and a complex alchemical device. The walls are lined with bookshelves and a single large chest sits against the north wall.
The chest, orrery and alchemical device are in fact mimics and will attack if they or the desk are searched. The mimics can be detected with a successful DC 30 perception check.
CR 7, 3,600XP
Searching the room turn up documents on the life cycles of Rust Monsters, theories on hybridisation of creature through magic, giantism and fast growth through the use of crystals.
2.9 The Fluid Tanks
Large glass tanks line the walls of this wide corridor. They are filled to the brim with a green viscous liquid. The north tanks are filled with dissected remains while the southern tanks have various sized brains bobbing in them.
A pressure pad (marked by the red square) will trigger a trap. A DC 20 perception check will discover the trap and it can be disarmed with a DC 25 Disable Device. If the trap is triggered the doors at either end of the room swing shut and lock. The tanks start draining and the fluids spill into the room. Each round a player is in the room, they must make a DC (13 + number of rounds in the room) Fortitude save or they are overcome by the fumes from the chemicals. If they fail, they are overcome as if affected by a Sleep spell.
The doors take a DC 30 Strength check to break open.
2.10 The Lab
Beneath the broken grate is a Gelatinous Cube that was used to dispose of waste. The Gelatinous Orbs have budded off from this and escaped via the drain. It’s well fed and happy to stay in there.
If the party didn’t encounter the Clockwork Soldiers in 2.3 Drain, they are here ransacking the room. They are armed with +1 Obsidian Halberds but are otherwise normal Clockwork Soldiers. If the party tripped the trap in 2.9 The Fluid Tanks, the Soldiers are prepared to engage the party.
CR 8, 4,800XP
2.11 The Apartment
This set of rooms is a well appointed apartment
A man dressed in wizard robes is slumped in one of the chairs slowly eating what appears to be a brain. He looks up and a vexed look crosses his face. “What now? Is there no end to this nonsense?” He stands up and there are wounds visible that should have someone struggling to stay alive, never mind stand.
The man is in fact possessed by the Intellect Devourer who was running the experiments in creating the Rust Dragon.
Once the host body takes 20 points of damage, the host will drop to its knees and the following happens:
“Well that’s not ideal”. The man drops to his knees. His mouth opens unnaturally wide and a large glistening brain on four spindly legs that end in claws crawls out.
When the Devourer reaches ~20 HP, it will attempt to flee through the secret door and trigger the rolling boulder trap to stop the party from following.
CR 8, 4,800XP
On searching the apartment, the following treasure is found in the chests:
As the Devourer escapes down the tunnel, they flip a lever and a boulder trap is activated. The boulder will block the entrance allowing the Devourer to escape. If the party manages to unblock the corridor, the stairs lead to a concealed exit in the forest above the mine entrance.
Postscript
As the party is leaving the mine, they are buffeted by gust of wind caused by the wings of a Clockwork Dragon landing in the camp. On his back is a ostentatious dressed figure. One of their arms is bare and is obviously mechanical.
“Well I suppose my tin soldiers don’t always work out. You’ve done their job for me so it’d be churlish of me to just kill you now. You will, of course, hand over any research or biological . Any trinkets you found are immaterial – maybe they’ll help you along your way.”
If the party looks like they are going to resist, a host of Clockwork Soldiers reveal themselves from the trees on the cliffs. It should be quickly apparent it would not end well for the party if they do not comply.
When importing a folder of images into Create ML, you may encounter the following error message:
Training Data Error
Empty table from specified data source
What this seems to actually mean is that there is something in the folder that is not:
an image
a JSON file with annotations for the images
If, for example, you are using RectLabel to annotate your images, there will also be an annotations folder with your images. You will need to remove this before the folder can be processed.
Once you’ve cleaned up your images folder, hit Rescan and they should be read properly and you can proceed to train your model.
One of my favourite sorts of twitter accounts like Random Bobbinsverse and Random Hellboy that tweet out single panels for a comic. They add a little bit of joy to my timeline between the tweets about our current Grimdark reality.
Recently I’ve had a need to look into creating Twitter Chatbots and creating one of these panel posting bots seemed like a good example project.
Why call the bot A Chip of Culture? Alan Moore is said to refer to floppy comic book issues “a real slab of culture” (although I can now only find references to Warren Ellis saying Moore said it). If a full comic is a “slab” then a single panel would be a “chip”.
To get the panel posting bot working you’ll need the following:
Python 3.7 (that’s what I have locally, may work on earlier versions YMMV)
Basic Bot
The core of the bot built off the back of the tutorial “How to Make a Twitter Bot in Python With Tweepy“. The tutorial results in a bot that can reply, favourite and follow. Once you’ve had a look at that, come back here and we’ll expand it to have a MySQL backend to hold some data and integrating with Dialog Flow to provide some smarts.
If you want to skip it, you can start with the code tagged “basicbot” in the repo for this bot.
Adding MySQL
Before we can interact with the database, the MySQL connector package needs to be installed:
pip install mysql-connector-python
To keep our boilerplate code for interacting with the database in the one place, we’ll create some helper functions around our calls – connect, fetchall, insert and update. You can see the code behind these in db/db_helper.py.
The database connection details should be set as environment variables for the connect function to use:
The first thing we want to store in the database is a list of comic panels that we will be posting. This table will contain all the information needed to post about a panel – who the creators are, what series, when it was published etc.. The only required field will be the id. Create the table using the following SQL:
image_uri (varchar): Location in filesystem of the panel image
twitter_media_id (varchar): Media id from Twitter after image is uploaded
writer (varchar): Who was the writer on the comic
artist (varchar): Who was the artist on the comic
letterer (varchar): Who was the letterer on the comic
colourist (varchar): Who was the colourist on the comic
inks (varchar): Who inked the comic
series (varchar): What series the panel appeared
issue (int): What issue the panel was in
published (varchar): When was the issue published
last_used (datetime): When the panel was last posted
number_of_uses (int): How many times the panel has been posted
We’ll create a class to map a row from the panels table into an object. This class will have a property which will generate the appropriate text describing the panel. As most of the columns are optional, this function will only include the relevant information.
When posting, we want to select a panel at random from the least used panels. This means that a panel will not be reused until all other panels have been shown. We’ll add a helper function called select_panel to perform the query, shuffle the results and return a Panel object.
def select_panel():
logger.info("Selecting a panel")
panel = None
sql_select_Query = "SELECT * FROM panels WHERE number_of_uses = (SELECT MIN(number_of_uses) FROM panels)"
records = fetchall(sql_select_Query)
logger.info(f"Number of records available: {len(records)}")
if len(records):
shuffle(records)
panel = Panel(records[0])
return panel
When a panel is posted, we want to store the media id of the panel image (so we don’t have to keep uploading the same image multiple times) and increment the use count. To do this, we’ll add a function called update to the panel class.
def update(self):
logger.info("Updating a panel to mark used")
self.number_of_uses = self.number_of_uses + 1
sql_update_query = """Update panels set number_of_uses = %s, twitter_media_id = %s where id = %s"""
inputData = (self.number_of_uses, self.twitter_media_id, self.id)
update(sql_update_query, inputData)
logger.info(f"Updated panel {self.id}")
Now that we can fetch a random panel and update that it has been used, let’s put it to use with a new script, autopostpanel.py.
#!/usr/bin/env python
# tweepy-bots/bots/autopostpanel.py
import tweepy
import logging
import time
import sys
import os
#This is so we can find out DB files
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/../')
from config import create_api
from db.panel import select_panel
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
def post_panel(api):
logger.info("Attempt to post a panel")
panel = select_panel()
media_ids = None
if panel:
tweet_text = panel.tweet_text
if not panel.twitter_media_id:
media = api.media_upload(panel.image_uri)
panel.twitter_media_id = media.media_id
media_ids=[panel.twitter_media_id]
else:
tweet_text = "No panel available!"
logger.info(f"Tweeting: {tweet_text}")
tweet = api.update_status(
status=tweet_text,
media_ids=media_ids
)
panel.update()
return tweet
def main():
try:
api = create_api()
post_panel(api)
except Exception as e:
logger.error("Error on post", exc_info=True)
if __name__ == "__main__":
main()
When this script is run, it fetches a random panel from the least used panels, uploads the panel (if it hasn’t been uploaded before), tweets it out and records the usage.
The updated code for this section is under the “postpanel” tag in the repo.
Recording Replies
In the current auto-reply script, it will forget what tweets have been replied to when it stops executing and will attempt to reply again on subsequent runs. This is less than ideal as it will only annoy people seeing the same thing over and over. To avoid this, we will record when a tweet has been replied to in the database.
Create the table to hold the tweet id of replied tweets with the following SQL:
To interact with this table, we will create two helper functions, last_reply to get the id of the last tweet that was replied to and did_reply to record that a tweet was replied to.
def have_replied(tweet_id):
logger.info(f"Checking if tweet {tweet_id} has already been replied to")
sql_select_Query = "SELECT * FROM replies WHERE tweet_id = %s LIMIT 1"
records = fetchall(sql_select_Query, (tweet_id,))
if records and len(records) > 0:
return True
return False
def did_reply(tweet_id):
logger.info("Adding a tweet id to mark it replied")
sql_insert_Query = "INSERT IGNORE INTO replies (tweet_id) VALUES (%s)"
insert(sql_insert_Query, (tweet_id,))
def last_reply():
logger.info("Get the id of the last tweet replied to")
sql_select_Query = "SELECT * FROM ellis_panel.replies ORDER BY id DESC LIMIT 1"
records = fetchall(sql_select_Query)
logger.info(f"Number of records available: {len(records)}")
if len(records) < 1:
return 1
return int(records[0]['tweet_id'])
The autoreply.py script needs to be updated to use these functions (see lines 37 and 43 below). For did_reply we pass through the id_str parameter of the tweet object rather than the 64-bit integer id.
#!/usr/bin/env python
# tweepy-bots/bots/autoreply.py
import tweepy
import logging
from config import create_api
import time
import sys
import os
#This is so we can find out DB files
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/../')
from db.replies import last_reply, did_reply
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
def check_mentions(api, keywords, since_id):
logger.info("Retrieving mentions")
new_since_id = since_id
for tweet in tweepy.Cursor(api.mentions_timeline,
since_id=since_id).items():
new_since_id = max(tweet.id, new_since_id)
if tweet.in_reply_to_status_id is not None:
continue
if any(keyword in tweet.text.lower() for keyword in keywords):
logger.info(f"Answering to {tweet.user.name}")
if not tweet.user.following:
tweet.user.follow()
api.update_status(
status="Please reach us via DM",
in_reply_to_status_id=tweet.id,
)
did_reply(tweet.id_str)
return new_since_id
def main():
api = create_api()
since_id = last_reply()
while True:
since_id = check_mentions(api, ["help", "support"], since_id)
logger.info("Waiting...")
time.sleep(60)
if __name__ == "__main__":
main()
When the script is run, it will ignore tweets that have already been replied to and avoid duplicates statuses (that would be rejected by the Twitter API anyway) and won’t annoy other users by bombarding them with the same response.
The updated code for this section is under the “recordreply” tag in the repo.
Integrating with Dialogflow
Now that we know when we’ve responded to a mention, let’s make our responses a little more intelligent by passing them through Dialogflow.
An agent needs to be created to mange the interactions. Hit ‘Create Agent’. Call the agent ‘ChipofCulture’ (no white space is allowed in the name) and hit ‘Create’. All the other options are configurable later in the settings page.
We now have an agent but it will only respond to the default questions. To correctly respond to mentions, we need to set up intents that will interpret what has been said and respond appropriately. The intent we will set up is to figure out when the bot has been asked to send a panel.
Click ‘Create Intent’. Call the intent ‘Random Panel’. Before the intent can be triggered, some example phrases need to be added. Based on these training phrases the agent will try and match was is sent to it and return the appropriate intent. Click ‘Add Training Phrase’. Enter ‘Show me a comic’ and ‘Send me a panel’. Hit ‘Save’.
You can test the intent by typing into the input box on the top right. If you enter ‘show me a panel’ (a combination of our two phrases) you will see that the correct intent is picked up.
The last piece is to add an action to the intent. this action will be used later in our scripts. Click ‘Add Parameters and Action’. Put ‘send_panel’ in the ‘Enter action name’ input box. Hit ‘Save’. Test the intent again and you’ll see the action is filled out.
If you hit the ‘Diagnostic Info’ button after testing your intent, you will see the raw JSON that will be retuned by the API.
We’re now ready to access the agent from our scripts. To open up API access, it must be turned on via the Google Cloud console. Once you are logged into the console, select APIs & Services. Click on ‘Enable APIs and Services’ and search for Dialog Flow and activate the API.
To access the API, you’ll need to create a service account to authenticate against. Download the json file and open it up. Find the key ‘client_email’ and note the generated address. Go to the Google Cloud console and click on ‘IAM’. If the e-mail address from your json file isn’t listed, click ‘add’. Enter the e-mail address in ‘New Member’ and for ‘select a role’, search for ‘Dialogflow API Client’ and click ‘Save’.
Once all this is setup, we need to install the Dialogflow package and other supporting Google packages for Python:
We now need to add a function that will take the text from a mention, pass it to Dialogflow and figure out what to do with the response.
#!/usr/bin/env python
# tweepy-bots/df/df_helper.py
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
def detect_intent_texts(project_id, session_id, text, language_code):
"""Returns the result of detect intent with texts as inputs.
Using the same `session_id` between requests allows continuation
of the conversation."""
import dialogflow_v2 as dialogflow
from google.protobuf.json_format import MessageToDict
session_client = dialogflow.SessionsClient()
session = session_client.session_path(project_id, session_id)
text_input = dialogflow.types.TextInput(text=text,
language_code=language_code)
query_input = dialogflow.types.QueryInput(text=text_input)
response = session_client.detect_intent(session=session,
query_input=query_input)
return MessageToDict(response.query_result)
The response from the API is turned from a protocol buffer object into JSON before being returned.
To use the function, we call it with our agent’s project id, a session key (we’re using a freshly generated UUID), the text of the tweet and a language encoding which we’ll take from the language set in the tweet.
You find the project id on the settings page accessed by clicking on the cog icon and will be stored as an environment variable:
export DIALOGFLOW_PROJECT_ID="your-project-id"
The UUID is if we want to expand out capabilities and be able to keep a session going with the agent. Using this is left as an exercise for the reader.
The autoreply.py script is updated with this function call. The result returned is checked for the send_panel action
def check_mentions(api, since_id):
logger.info("Retrieving mentions")
new_since_id = since_id
for tweet in tweepy.Cursor(api.mentions_timeline,
since_id=since_id).items():
new_since_id = max(tweet.id, new_since_id)
if tweet.in_reply_to_status_id is not None:
continue
result = detect_intent_texts(dialogflow_project_key, str(uuid.uuid4()), tweet.text, tweet.lang)
if result['action'] == 'send_panel':
logger.info(f"Answering to {tweet.user.name}")
if not tweet.user.following:
tweet.user.follow()
post_panel(api, username=tweet.user.screen_name, reply_tweet_id=tweet.id)
did_reply(tweet.id_str)
return new_since_id
Based then on the action set in the response, we can do the appropriate thing, which here is to send a random comic panel.
The post_panel function in autopostpanel.py has changed slightly, to accept an optional Twitter handle and tweet id to use when posting a tweet. This allows for a threaded reply rather than just a random dangling tweet.
def post_panel(api, username=None, reply_tweet_id=None):
logger.info("Attempt to post a panel")
panel = select_panel()
media_ids = None
if panel:
tweet_text = panel.tweet_text
if not panel.twitter_media_id:
media = api.media_upload(panel.image_uri)
panel.twitter_media_id = media.media_id
media_ids=[panel.twitter_media_id]
else:
tweet_text = "No panel available!"
if username:
tweet_text = f"@{username} " + tweet_text
logger.info(f"Tweeting: {tweet_text}")
tweet = api.update_status(
status=tweet_text,
media_ids=media_ids,
in_reply_to_status_id=reply_tweet_id,
)
panel.update()
return tweet
Now when there is a tweet in the bot’s mentions that asks for a panel, the bot will reply directly to that tweet.
The updated code for this section is under the “adddialogflow” tag in the repo.
Conclusion
In this tutorial, we’ve expanded on a basic Twitter bot to add some state so it remembers what has been replied and given it some smarts to reply sensibly.
More intents can be easily added to the Dialogflow agent so you can respond to other questions such as asking the bot what artists does it know about or to send a panel from a particular series.
Given my penchant for calculators and the fact that I’ve a Property Price Register site, the above tweet came as no surprise and I really should have knocked something together myself as soon as I saw the news.
So without further ado, here’s a quick and dirty calculator to work out how much of a deposit you need under the new rules:
Luckily, this won’t effect me for another few years as I managed to finally close on my apartment last week but there are plenty of folks for which the possibly of a 20% deposit was a nightmare scenario.
CBPWordPress is an iOS library that will allow you to easily include content from a WordPress blog. The library can fetch lists of posts, individual posts and submit comments.
It is, of course, available on GitHub under the MIT License as well as via CocoaPods.
Background
Over the last few years, I’ve built an maintained the accompanying iOS app for Broadsheet.ie. It’s nearly tradition now for me to release a new version every year
The Example App
Included in the repo is an example app that is the basis for the new Broadsheet.ie app. This app will allow you to browse the site content, search for posts and submit comments.
The app also refreshes itself in the background, reminds the user of its existence in the morning and evenings and updates the logo on the home screen every 6 hours.
Before you can use CBPWordPress, you must install the WP-JSON-APIplugin to provide the data to the app. This is a slightly modified fork that provides a few extra fields to make things a bit smoother for the app.
If you want to be able to submit comments via the app, remember to turn on the Respond controller in the JSON API settings section.
Getting Started With The Example
Check out the repo from GitHub and initialise the Example pods:
git clone https://github.com/kmonaghan/CBPWordPress.git
cd CBPWordPress/Example
pod install
If you open the example project workspace and build and run the app, it should pull the latest 10 posts from Broadsheet.ie.
Using CBPWordPress In Your Own App
CocoaPods
The easiest way to use CBPWordPress is to install it via CocoaPods. To do that, just add the following line to your Podfile and then run ‘pod update’.
pod 'CBPWordPress'
Add the CBPWordPress project to your project
If you don’t use CocoaPods, add all the files in the CBPWordPress folder to your project. Then simply include the CBPWordPress.h header file where you want to use the library.
Note that you will also have to include AFNetworking in your project.
Usage
Pointing the library at your own WordPress installation is trivial. The first call to the library should be to set the root URL for the API.
count: The number of posts to retrieve. Defaults to 10.
Fetching A Post
If you know the post id, you can fetch a post using the fetchPostWithId:withBlock: method from the NSURLSessionDataTask category. The example below fetches post 1234 and assigns it to a local post variable.
If you have the URL of the post, you can use the fetchPostWithURL:withBlock: method instead. You pass the full URL of the post as the parameter.
Comment On A Post
To comment on a post, the postComment:withBlock: method from the NSURLSessionDataTask category is used. The method takes a CBPWordPressComment object as the first parameter. Below is an example comment being initialised.
CBPWordPressComment *newComment = [CBPWordPressComment new];
newComment.postId = 1234;
newComment.email = @"example@example.com";
newComment.name = @"Jonny Appleseed";
newComment.content = @"This is a comment!";
//Optional
newComment.url = @"http://somewebsite.com";
//If the comment is replying to another comment
newComment.parent = 1234;
Note that the URL and parent properties are optional but everything else is required. The parent property should be only be set if the user is replying to a comment and should be that comment’s id.
Once the comment is initialised, pass it to the postComment:withBlock: method. In the following example, the new comment is submitted and on success is set to the returned comment object.
Monday was the 4th birthday of Broadsheet and as is tradition, I bring you some stats from the last year. As usual, these all come from Google Analytics and cover from 29th of July 2013 to 28th of July 2014.
The Headline Figures
New users up 3% to 24% of visitors
Users up 6.5% to 3.2m from 3m
Pagviews down 14.5% from 32.5m to 27.8m
Screenviews stand at 20.5m
So why the drop in pageviews?
It looks like the mobile apps are cannibalising the desktop site.
This is in part due to the an Android app finally getting released, but also because of the ongoing trend across the web of the growth in traffic from mobile apps.
I don’t have a full year of stats for the 2012/2013 to compare mobile usage with but with a bit of hand wavying estimations based on the 5 months of data I do have, there’s about a 25% increase in screenviews. If I combined the 20.5m screenviews (which are setup to be the equivalent of a pageview on the desktop site) between the iOS and Android apps to the desktop, you get about 12% growth year on year.
Outside of the apps while the traffic from mobile has increased by 5% to 16% of the total pageviews to 3.3m. Tablet usage just increased 1% to 793K.
iOS powered devices still deliver 2 pageviews to every 1 from an Android device.
Samsung dominates the top 50 devices used with 23 models (although some are variants on the same handset). Sony has 6, HTC weighs in with 5, Apple and Nokia have 4 each and Google just 2.
What Are You Reading?
As has always been the case, about half of all pageviews are the front page of the site. The top posts more represent what people have commented on or shared, since the stories are published in full when you’re looking at the home page.
11,464 post were published over the course of the last year. The top five by pageviews were:
Interestingly, three of the stories (‘The Mask Is Off And People Know’,Who Is He? and Dear RTÉ) were only published in the last week, while What Your Electric Guitar Says About You is nearly two years old. These are both perfect examples of the two ways posts become huge. The recent stories are (obviously) very current and immediate. People are sharing and commenting one them as they’re igniting people’s passions. The guitar post, on the other hand, is one that has enduring interest and every so often gets spread around forums and gets shared on Facebook again and again.
The fact that four of the top five posts are rather serious content (a trend that extends to the top 25 posts of the year) is a possible indication of a more serious tone on the site or that people are more inclined to share the more serious material.
What Are You Riled Up About?
There’s been 192,170 comments from 10,117 commenters. Of these, 21 commenters have posted over 1,000 comments each and responsible for 45,688 between them.
62% of the referral traffic comes from Facebook alone. They have firmly placed themselves as the distribution channel of choice for many people. Each change they make to how posts appear in news feeds has a massive and very visible impact. The last few changes have been massively detrimental to site like Broadsheet that can’t afford to pay Facebook to reach people who’ve already ‘liked’ the Broadsheet feed.
Twitter takes up 23% of the remaining referral traffic, which is less than the traffic coming from just the Facebook Mobile site.
Everything after the big two provide only tiny scraps of traffic.
The top five search terms (with the usual variations on broadsheet.ie removed) that brought people to the site were:
teletubbies
justin timberlake phoenix park
vikings
forbiddem fruit 2012 pictures
judge nolan
I’m at a loss to explain the teletubbies one. After three years appearing in the top five, ‘niamh horan’ has slipped down to sixth. I almost feel like it’s an end of an era.
The Window You Look At The Internet Through
Chrome is the absolute king, with 44.5% (up 6%) of all pages viewed on it. this gain was mainly at the expense of both Internet Explorer (dropped 3% to 14.5%) and Firefox (down 4% to 17%). Safari stayed relatively stable at 14% (a 0.5% drop).
The other browser that gained share was the Safari in-app browser (i.e. if you view Broadsheet from with an app like Facebook or Tweetbot etc.) which is up 2% to 5%.
IE 6 and 7 are near extinction, with less than 1/2% of pagesviews from them. IE 8 though is still the most used of those browsers, although it is declining in favour of IE 10 and 11.
When it comes to whats powering the machines people are using, Windows still holds 50% (albeit that’s down 9%). iOS has overtaken OS X for second place with 21% (up 9%) compared to 15.5% (down 5%) followed by Android which has doubled to 10%. The rest is made up of a smattering of Linux, Chrome OS, Symbian Os and Windows Phone.
Mobile Apps
5,325 (for a total of 29,901) on iOS
4,555 (for a total of 6,819) on Android
1,829 (for a total of 3,200) on Windows Phone
OS wise, only about 6% of installs are on a version of Android under 4.0, which is great news for me as I can aim the app to be 4.0+ plus in the next iteration. The downloads for it are a little disappointing but on the flip side, the app isn’t great and does next a bit of love and attention.
On iOS, less than 4% of active users in the last month had some version of iOS 6 on their device. Again, this is great new for me as it means I can drop support for 6 with minimal impact.
Where Are You Based These Days?
Unsurprisingly, there’s no change here. Visitors predominately come from Ireland (74%) followed by the UK (10%), US (4%), Germany (1%), Australia (1%) and then everyone else makes up the other 10%.
Anything Else?
If there’s anything else you’d like to know about, ask in the comments and I’ll see what I can dig out.
There’s been a lot of interest in the office to see what the inside of the The Three Tun Tavern was actually like since they started renovating it a few months ago. Since today was the official opening, 10 of us headed down for lunch out of sheer nosiness.
Inside it seemed brighter and airier than the previous incarnation, Tonic. The bar has been moved from the center of the building to along the back wall, opening it up a lot more space. Some of the internal walls have been removed as well so there’s more overlooking the central area. The over all impression though wasn’t of a brand new pub, rather one relatively recently renovated but lived in.
There were an awful lot of staff on but it’s not clear if that’s just because it was opening day or will be the norm. They were certainly needed today as there were a constant stream of customers coming in, as well as a few curious people just having a nose around.
Chicken & Ribs Combo
Sirloin of Rump Steak
Food wise, the menu is rather extensive albeit standard pub grub fare. They’ve various food ‘clubs’ on during the week with themed specials.
Being lazy and since it was their Tuesday Steak Club, I (and most of the rest of the guys) ended up having a rump steak and chips for €9.95 (it’s normally €12.95). A few more chips wouldn’t have gone astray and the steak was a tiny bit overcooked for rare but the meat was flavourful. Considering the price (which included a soft drink) it was good value.
It was opening day though, so there was the inevitable hiccup with one of the meals which left one of the guys waiting 30 minutes for his veggie burger. It’s probably a little unfair to judge them on this though since it was opening day.
Overall the consensus was that the food was fine and filling but nothing spectacular. It’s another lunchtime option for Blackrock, but you wouldn’t be going out of your way to eat there.
I will be back at least once more though as they’ve a good selection of craft beers on tap that I’d like to try (and retry Hobgoblin to see if my college days impression of it being vile is true).
Okay so it’s a Wetherspoon’s pub but to be honest, it’s not particularly different to many pubs already operating here. It’s probably going to do a decent lunchtime trade from the off but I wouldn’t like to make predictions about the evening.
One of the exciting new things announced at the WWDC keynote this month was App Extensions. This really opens up inter-app communication as well as letting developers do interesting things with the notification center and sharing.
I’m convinced that pretty much every news organisation is going have a widget included in their app to show the latest couple of stories. To that end, I’m going show here how I added a Today widget to a simplified version of the Broadsheet app.
This all works in the simulator but has not been tested on a device yet Kindly tested by Liam Dunne and it works (phew!)
Some of the cell layouts are a bit off but are good enough for informational purposes
This is still all in Obj-C rather than being all cool and done in Swift
Starting off
Our starting point is a simple app that grabs the JSON feed of the latest 10 postsBroadsheet.ie and displays them in a UITableView. Tapping on a cell brings you to a screen that displays the content of the post in a UITextView (this will sometimes be mangled due to the content).
To this app we’re going to add a Today widget that displays the latest two post and brings you back to the app when a cell is tapped.
The base app is in the (imaginatively called) ‘baseapp‘ branch.
Creating The Framework
First, we’re going to create a framework to hold the common files that will be used by both the app and the widget. We need to add a framework target to the project by doing the following:
Select the project
Editor->Add Target
Select ‘Framework & Library’
Select ‘Cocoa Touch Framework’ and hit ‘Next’
Give it a name (ours is called CBPKit) and hit ‘Finish’
The framework should be automatically included in your app target (check under ‘Link Binary With Libraries’ in ‘Build Phases’).
Move any files you want your framework to provide from your app to the framework (note that this doesn’t move the files on disk). In this case it is everything but the view controllers.
Select the headers and in the ‘Utilities’ pane select your framework and set the option on the right to ‘project’.
Select the implementation files and again in the ‘Utilities’ pane select your framework and unselect your project.
For displaying the images in the cells, I’m using SDWebImage. I could have just add the relevant SDWebImage files to the framework I just created, but this seems wrong to me as:
You may include another framework in the future that includes and exposes SDWebImage and then you’d have a clash
You release your framework and someone can’t use it because your included version of SDWebImage clashes with the version they want to use
SDWebImage may release its own framework in the future
Instead, I created a separate framework (with the exact same steps as above) called SDWebImageKit.
As I may want to use this framework in another project, I set the headers to ‘Public’ in the ‘Utilities’ pane and added each of the headers to the framework header file. This means that I can import all the headers at once using:
#import <SDWebImageKit/SDWebImageKit.h>
When you run the app now, it should look no different to before despite the internal changes.
If you switch to the ‘framework‘ branch, you’ll see this built on top of the previous branch.
Today Extension
Like the framework, we first add a Extension target to the project.
Select the project
Editor->Add Target
Select ‘Application Extension’
Select ‘Today Extension’ and hit ‘Next’
Put in a name (ours is called CBPTodayExtensionExample) and hit ‘Next’
Go into Build Phases and include both the frameworks from above in the ‘Link Binary With Libraries’
If you run the project now in the simulator and pull down the notifications drawer, you should see ‘1 New Widget Available’ at the very bottom. Tapping on ‘Edit’ will bring up all the available widgets and the new widget should be at the bottom with a green plus beside it. Tap the plus and it should be displayed with any other active widgets when you hit ‘Done’.
‘Hello world’ is a nice greeting but we want to replace this with a table containing 2 tappable cells containing the latest two posts from Broadsheet.
I don’t use storyboards, so I removed the default one added by the template. In info.plist I removed the key NSExtensionMainStoryboard and replaced it with NSExtensionPrincipalClass and put ‘TodayViewController’ as the value.
The cell I use in the app is a bit big for displaying in the notification center, so I added a set of new constraints to display a more compact cell. This means I can reuse the same data source and cell from the main app in my Today extension with just some minor modifications.
In the TodayViewController, there are two places that need to load data from the network – when the widget is created and when widgetPerformUpdateWithCompletionHandler is called. For the former, I load posts in viewDidLoad, so that they should be ready by the time the widget displays. When iOS thinks the widget will be displayed to the user after it has been first displayed, widgetPerformUpdateWithCompletionHandler is called giving the widget a chance to update the posts displayed.
In this example, both cells are always reloaded even if they already contain the posts returned. It probably shouldn’t do this and instead only replace the updated cells, but that’s a bit of extra polish I’ll leave for when building the full widget.
To get the app to open when a cell is tapped, a custom URL scheme is registered in the base app and this can be called using the UIViewController’s extensionContext to open the URL.
If you switch to the ‘todayextension‘ branch, you’ll see this built on top of the framework branch.
Finally
To keep everything tidy (and to satisfy my developer OCD), I used Synx to synchronise the files and folders to match those used in the project.
That’s all that’s needed to get a Today widget up and running. I’m delighted it’s so straight forward and I’m looking forward to seeing what people do with them.
Aside
There’s an issue with the debugger in Xcode attaching to the extension process properly. You can manually attach it by selecting ‘Debug’->’Attach to Process’ and finding the extension process under ‘System’. If your extension crashes on startup you won’t be able to this.
If you want to see logging from before you reattached the debugger, you can view it in the system log. You can open it from ‘Debug’->’Open System Log…’. If you filter it by your extension’s bundle id you’ll get the relevant entries.
Update 18/6/2014: In the release notes for Xcode 6 Beta 2 there’s a list of issues around the debugger and extensions. One pertinent to this app is that you can’t debug a today widget in the simulator but you can on device.
Update 7/7/2014: Xcode 6 Beta 3 came out today and fixes some of the debugging issues.
I spent a rather frustrating few hours this morning trying to get push notifications set up on Amazon’s Simple Notification Service.
In the Getting Started with APNS, there is a section on converting the p12 formatted file exported from the key chain and converting it into pem format.
If you try and use this pem file in the management console, the following error is returned:
File type is not supported or file has been corrupted
What you actually need to upload is the p12 version.
