Recent Revsys Updates

So yeah it's been awhile since we've had a chance to blog, but [insert witty bit about too busy getting stuff done here]... However, we have had a few great things going on in the recent months that I want to highlight.

Since we acquired Grove last december we've been plinking away at some bugs, performance issues, and of course the stability problems that plagued it. I'm happy to report that the performance and stability issues have been resolved along with a few feature enhancements and other random bugs.

As usual, we sponsored PyCon and this year even had a kick ass booth that we shared with our buddies over at Project Evolution who we're working with on the much anticipated redesign of Python.org. We also held a kick ass private party, but look for an upcoming blog post on those shennanigans.

Among our many client projects, two we can talk about publicly. We recently refreshed the Nielson Norman Group website and launched a whole new backend for the Sony website.

We also brought on another new team member, Flavio Curella who we managed to snatch up with the desmise of the software division of the company that originally created Django where most of us worked at one point.

We also put up an updated version of our popular Django Cheatsheet updated for version 1.5.

And to top it all off, we launched a low traffic Django Tips and Tricks Newsletter. We promise not spam the crap out of you.

Until next time!

Posted on April 25th, 2013 at 14:20:39 and has 1 comments

Grove isn't dead, it's only resting

The news is out: we acquired Grove today, and we're taking over operations immediately.

We think Grove is an incredibly cool tool: it's an IRC server masquarading as a web-based chat application. It's got a really slick web UI that anyone can use, but behind the scenes is IRC, a proven technology trusted by über-geeks everywhere. We love the way Grove bridges the new and the old.

We're first and foremost Grove users, so we think we've got a good grasp of what Grove's other users want. In the short term our focus is going to be on fixing connectivity and improving site performance, but after that we've got some big ideas we're planning to implement. And of course we'd love to hear your ideas for what Grove should do; let us know at team@grove.io.

We're really glad that we could give Grove a home at Revsys. We think it's a great match for our company, and we can't wait to start working on our big ideas.

If you're already a Grove user, I hope you're as excited as we are to hear that it's got a bright future. And if you're not already using Grove, why not check it out? We think you'll love it as much as we do.

Posted on September 28th, 2012 at 11:54:28 and has 0 comments

Three things you should never put in your database

As I've said in a few talks, the best way to improve your systems is by first not doing "dumb things". I don't mean you or your development staff is "dumb", it's easy to overlook the implications of these types of decisions and not realize how bad they are for maintainability let alone scaling. As a consultant I see this stuff all of the time and I have yet to ever see it work out well for anyone.

Images, files, and binary data

Your database supports BLOBs so it must be a good idea to shove your files in there right? No it isn't! Hell it isn't even very convenient to use with many DB language bindings.

There are a few of problems with storing files in your database:

• read/write to a DB is always slower than a filesystem
• your DB backups grow to be huge and more time consuming
• access to the files now requires going through your app and DB layers

The last two are the real killers. Storing your thumbnail images in your database? Great now you can't use nginx or another lightweight web server to serve them up.

Do yourself a favor and store a simple relative path to your files on disk in the database or use something like S3 or any CDN instead.

Ephemeral data

Usage statistics, metrics, GPS locations, session data anything that is only useful to you for a short period of time or frequently changes. If you find yourself DELETEing an hour, day, or weeks worth of some table with a cron job, you're using the wrong tool for the job.

Use redis, statsd/graphite, Riak anything else that is better suited to that type of work load. The same advice goes for aggregations of ephemeral data that doesn't live for very long.

Sure it's possible to use a backhoe to plant some tomatoes in the garden, but it's far faster to grab the shovel in the garage than schedule time with a backhoe and have it arrive at your place and dig. Use the right tool(s) for the job at hand.

Logs

This one seems ok on the surface and the "I might need to use a complex query on them at some point in the future" argument seems to win people over. Storing your logs in a database isn't a HORRIBLE idea, but storing them in the same database as your other production data is.

Maybe you're conservative with your logging and only emit one log line per web request normally. That is still generating a log INSERT for every action on your site that is competing for resources that your users could be using. Turn up your logging to a verbose or debug level and watch your production database catch on fire!

Instead use something like Splunk, Loggly or plain old rotating flat files for your logs. The few times you need to inspect them in odd ways, even to the point of having to write a bit of code to find your answers, is easily outweighed by the constant resources it puts on your system.

But wait, you're a unique snowflake and your problem is SO different that it's ok for you to do one of these three. No you aren't and no it really isn't. Trust me.

Posted on May 1st, 2012 at 16:24:03 and has 33 comments

PEP712 - Proposal to make unittest2 more accurate

Introduction

This PEP describes a proposal to make unittest2 output more accurate and fun.

The Proposed Solution

Upon having more than 7 failing tests in a test run replace all 'F' character output with 'U's for the remainder of the test run.

Rationale

This output formatting more accurately describes the mental state of the developer.

Example

.....FFFFFFFUUUUUUUUUUUUUUUUU
CK
----------------------------------------------------------------------
Ran 25 tests in 3.14159s
FAILED (failures=19)

Optional Add-On

Replace next passing test after 19 failures with 'CK'.

Reference Implementations

See Reddit.com

References

@f7u12
F7U12

Posted on October 20th, 2011 at 14:35:56 and has 3 comments

longjmp() 2011 - Impromptu Party

Lawrence has been having a bunch of downtown block parties lately, which are always great fun. The last couple of days the Kansas Relays have been going on and the long jump event was setup directly outside our office windows. Since we had such a great view we quickly through together a little party and invited all of our geeky friends to come watch, we had a pretty good turn out (free beer works apparently). Here are some photos I took from our office. You can find more professional quality photos of the event here.

Thanks to everyone for coming out!

Posted on April 22nd, 2011 at 13:30:35 and has 0 comments