Not waving but drowning

Today was the second of a series of hack mornings at the Think Tank Congress coworking space. The results were even more stunning than the first time, and sparked a lot of ideas. Here is a summary of the projects that were completed in 2.5 hours

• John Colton joined us as a C hacker looking to get his feet wet with web development. By the end of the morning he got a python web app running with Flask, got set up with Github, and deployed the app to Heroku.
• Abe Fettig, Alex Launi, and Elliot Murphy worked on a web enabled door latch with a Raspberry Pi, using a Node.js REST API talking to a PiFace digital I/O board. At the end of the morning they demoed pressing a button on a single-page web app served from Heroku and the local door latch buzzed and opened. Next time additional sensors and some real time status updates will be added, and then it’s time to think about installing it on a real door.
• Patrick Kenney was exploring building a responsive 3 column single-page web app, rendering some of the areas off-canvas and then using ScrollTo for navigation and pure CSS3 animations. It worked and he published the code!
• Alex Dorsk was working on a web app which aggregates information about all the music events happening in Portland. Since each venue publishes the calendar in a different way, he needed to do screen scraping, and Alex got scrapers written for 7 venues in a single morning using ScraPy.
• Hasan Adil was experimenting with data driven visualization using D3.js. He got an interactive force directed graph running, visualizing node size and connectedness. Shiny!
• Daniel Kopyc added speech recognition to an iOS app, using the OpenEars CMU speech recognition library, and drove his demo by talking to an iPad.
• Hugh Morgenbesser put together a Facebook app using Django. He showed how using 2scoops of Django, Fandjango, and FacePy make it possible to have a running app in a couple of hours.
• Alex Jones is working on digitizing beekeeping, and wanted to see whether he could use computer vision to count the number of mites on an IPM board. Using Processing PYImage and a digital photo of a board with some mites on it, he was able to “see” all the mites using color and blob detection and then calculate the number of mites by estimating the average area of a typical mite.

It was an impressive and energizing array of projects. Thanks to Patrick Roche for hosting the event at Think Tank, Abe Fettig for organizing, Hugh Morgenbesser for donuts, and Hasan Adil for coffee. At the end of the morning, we realized it would have been easier if we were using a shared document to gather links and instructions together over the course of the morning instead of trying to gather the details at the end, and we also struggled with the presentations due to poor WiFi coverage in the front room, something we’ll be trying to correct for next time. Thanks to everyone for coming out!

Raw materials for the next hack morning. We are working on web-enabling the door latch at Think Tank so that we can open the door remotely from a web page. We’ll be using a Node.js REST API connected to the GPIO pins on a WiFi-enabled Raspberry Pi. The GPIO pins operate a relay which operates a door latch with a builtin electromagnetic release.

Raspberry Pi with relay and electromagnetic door latch.

Today we had the first of a series of hack mornings at the Think Tank Congress coworking space where I spend most of my working time. Abe Fettig organized it, and even arranged coffee from Speckled Ax and potato donuts from Holy Donut as well as giving away of few copies of Programming Twisted, 2nd Edition. The idea of a hack morning is to take a break from your normal projects and work on something new or experimental, and then do a 3 minute demo of what you learned/accomplished at the end. Working on the ideas together in the same room with a ticking clock is a nice change of pace and encourages a lot of information sharing. We had coffee at 8:30, worked until 11:30, and then did a bunch of demos. There were 6 of us that participated, and another half dozen that stopped by to watch the demos, so it was a lively morning.

Photo credit Jason Sandifer

I had been itching to get a Geckoboard custom map widget to draw a map on a dashboard using data from my own JSON API, and last night Daniel Piltch of Freeport Metrics offered to bring over a new Raspberry Pi, so we teamed up and built a kiosk. The Raspberry Pi is a cheap, tiny computer with no moving parts or fans that runs linux. We managed to get the Raspberry Pi to boot directly into Google Chromium running in kiosk mode and loading a Geckboard dashboard with my custom latitude/longitude points displayed along with the event schedule for Think Tank pulled from Google Calendar, all displayed on a big HDMI monitor. I can’t wait to do more with the Raspberry Pi, it is a remarkable little device.

Hugh Morgenbesser of TripAdvisor worked on a Django project where he built a web UI that allowed people to enter a street address, and then he used the Google mapping APIs to geocode the address into latitude/longitude points and store the whole thing in a database, and then plotted the markers on a map. It was cool to see this, it’s the very next step I need for my own mapping project. Continuing on the mapping theme, Patrick Kenney of Deviceable worked on a business model canvas and a Rails app which he described as “Pinterest for Modern Architecture & Art”. This took mapping local: the idea is to enable people to share interesting/noteworthy buildings, art installations, and other points of interest, and then discover what is nearby based on your location.

Hasan Adil of LabelTop Software decided to try Ember.js. He has been building a project management application for iOS, and wanted to expand into making a web version. He managed to get a TODO list app running all in Ember.js, and showed off the polished interactions. Hasan described his approach as opposite from lowest-common-denominator thinking – he is building a whole new codebase for the web version because wants his product to take full advantage of the strengths of each different platform.

Abe Fettig of Fig.ly planned to get his hands dirty with drawing and graphics, and exercise an idea about music recommendations based on longitudinal analysis of listening habits – don’t just consider what you have rated as 5 stars, but figure out e.g White Stripes are more of a summer band than a winter band. He managed to pull together graphs of listening frequency over time for his whole iTunes library from the last few years, and plans to add in data from Spotify & Rdio next. He plotted against the seasons and it was interesting to see that there were some seasonal patterns in a few of the bands.

Overall this was a lot of fun, and definitely worth the effort of carving out a morning to do it. I’m looking forward to the next one!

Heroku is a convenient hosting solution for your web application, it provides a great deal of automated infrastructure so that you can deploy with a single command. Heroku provides a bunch of add-on resources such as databases, cache servers, etc. Here is a nice article about getting started with deploying an application on Heroku.

When you initially get your application running, you aren’t very likely to forget about the database. After all, your application probably won’t run without the database. If you are feeling proactive, you might set up some caching such as memcache or redis. But there is a lot more to worry about when you go from a proof-of-concept application to something running in production with customers and revenue depending on it. Here is an incomplete checklist to see if your app is ready for production.

1. A staging environment. It’s great that you can deploy code in a single command, but it’s worth registering a second heroku application and making that your staging environment so that you can do more in-depth testing of major changes before deploying them to production. Even better, sign up for a CI solution such as Tddium, and have it auto-deploy to your staging environment once unit tests pass. Don’t forget about cross-browser and cross-device testing for the most critical parts of your site – SauceLabs is great for this and can be wired right into your test suite.
2. Storing logs. The heroku logs command is very helpful, but those logs aren’t stored very long. You need something like logentries or papertrail to store your web logs for a longer period of time.
3. Tracking exceptions. While developing your application, there will be bugs. What do you do when the logs show that you have a bunch of 500 errors? You need application exception tracking and notification. This allows you to monitor for crashes in your app and fix them. Most crashes are never reported by customers, you need to know when they happen and fix them. I like exceptional.
4. Sending email. When you try to do your first password reset, you’ll quickly realize that the servers need to be able to send email. Transactional emails are different from marketing/list emails, and you need to be able to get reports on what emails were delivered and what emails had problems. Sendgrid has worked very well for me for transactional emails, and Mailchimp for newsletter style emails.
5. Restores. Notice how I didn’t say backups? You need to regularly test that you can restore your database(s) for use with in a different deployment. I recommend doing a database restore from production into staging regularly, so that you know it works. You will be surprised at how many things break the first time you try and do this, and will sleep better when you know that your restore/recovery process is regularly being exercised.
6. User behavior monitoring. Is anyone using your site? What pages are popular? Where does your traffic come from? Google Analytics is the default choice, but also consider KISSMetrics, they can provide valuable insights into what people really do with your application.
7. Performance monitoring. Nobody writes slow code on purpose, but everyone is horrified when they look at NewRelic for the first time and see what is really taking up the time and memory in their application. It’s expensive, but use it for at least a month – much cheaper than an engineer spending hours optimizing the wrong thing. NewRelic can also alert you when performance or error rates cross a threshold, so you know when your site is being overwhelmed before it crashes.
8. Availability monitoring. How do you know when Heroku has an outage? What if Heroku has an outage affecting only 1% of their applications – are you affected? What if that cool search provider you added a dependency on has an outage that means everyone trying to use your site sees error messages in the sidebar? If you don’t have something more sophisticated in place, at the very least set up pingdom to monitor your site uptime. It’s also a good idea to deliberately break your site by disabling each of your add-ons one by one and observing the failure mode – you may be able to handle partial failures more gracefully with very simple changes. Once you’ve done that, add some very simple hidden pages that run a check against each add-on and report whether it worked or didn’t work, then connect each of those pages into your monitoring system so that you can know when one of your add-ons had an outage even if your main application stayed up.
9. Notification and escalation. My two favorite ways to make progress on solving problems and doing creative work are to go for walks away from my computer and to sleep. That means I often won’t see the emails or chatroom notifications from those monitoring systems for several hours. You need something to text you and call your phone when a bad problem happens, and to escalate to the next person in the response team if you don’t acknowledge the alert within a certain amount of time. Pagerduty is the gold standard here, although OpsGenie is an interesting competitor.
10. Security vulnerability notifications. When a serious security vulnerability is disclosed, you need to respond quickly to patch the holes in your application. Look through your development stack, and figure out what the security announcement mailing lists are, and subscribe to them. Finding out a day or two before the vulnerability is in the news can make a big difference in whether you get things fixed before someone comes and downloads your entire database.
11. File storage and transcoding. Whether it’s something as simple as user profile pictures or videos giving a tour of your product, you won’t be able to store the files on Heroku. Amazon S3 is the most common choice here, and there are also services like Zencoder that can handle processing and transcoding videos and audio files into your preferred format before you store them.
12. SSL. Serve everything over SSL. Don’t let your certificates expire (set reminders or wire certificate checks into your monitoring system. Make sure your staging systems use SSL also.
13. Payment processing. Businesses need to charge money for goods and services, and even charities need to accept donations via credit card. I like Recur.ly for recurring billing – the javascript forms fit nicely into your own site, and still offload the bulk of PCI compliance headaches to the provider. They also take care of Dunning.
14. User Feedback. If you really want to engage your customers, give them a way to talk back to you. If you want to send your customers email, why wouldn’t you also give them an easy way to send you a comment or question? UserVoice is a great tool for this.
15. DNS. Use DNSimple for your app hosted on Heroku. Here’s why.
16. Search. Your users do not know how your site is organized, and don’t want to look at a site map. ElasticSearch is my default choice for search, and there are several add-on providers that will host your ElasticSearch cluster – I’m using Found and have gotten excellent service.
17. We covered backing up the site, but don’t forget to back up your laptop. You don’t want to be crippled and unable to access production systems when your laptop is run over by a taxi. I like CrashPlan because it is cross platform, backs up both locally and to the cloud, and warns you when backups haven’t been working. Another approach is to use a cloud development environment such as Cloud 9 IDE – but don’t forget, they can have downtime too.

When you’ve got your app up and running and are building your feature list for launch, don’t forget all these other things. They are the difference between a professional production quality application and a prototype. Did I miss something, or is there a better option? Please let me know.

Google Apps was used at the last 3 companies I have been involved in. At one company, we had a bad problem with Google Apps. If this problem happens to you it could get you in big trouble. While doing some auditing and security work, I discovered a second bad problem with Google Apps.

Problem number 1: You need backups. Google’s highly redundant self healing magical SaaS technology does not protect you from yourself, it only protects you from Google. If you or someone at your company accidentally or maliciously deletes a bunch of important business documents, they are gone. Google cannot help you get those documents back. If you think about this for a minute, it makes sense – if Google held onto confidential business data after you ordered them to delete it, nobody would want to use them. Nearly everyone I talk to who is using Google Apps overlooks the fact that they need backups, and are vulnerable to severe data loss. There are add-on services for Google apps that solve this problem, and provide you with auditing capability for the backups – I have used Backupify at two companies and it’s worth every penny. Employees can then self-service restore their own data, and administrators can recover data as well.

Problem number 2: If you have more than 1 person enrolled in your Google Apps organization, it is impossible to audit how documents are shared. Sharing documents with Google Apps is very easy, but when you’ve been using it for a couple of years nobody will remember what they have shared and whether it that level of sharing is still appropriate. Discovering that confidential financial information or trade secrets are accidentally shared with the public could be a major security breach. Recently I started using another add-on called CloudLock. CloudLock will scan your organization and generate reports of what is being shared, as well as run a content policy engine to warn you if there are documents that contain sensitive information such as credit card numbers, social security numbers, or other information that needs to be carefully safeguarded. Often data misuse is accidental rather than malicious. Not performing audits is sloppy and can get you big fines and/or fired.

Setting up backups, audits, and content policy checking only takes about 30 minutes and dramatically changes your risk profile for using Google Apps. Do it today.