This post is less about the
basics of counter_cache support in Rails
and more about best practices for introducing a new counter cache to an
existing project. More specifically, the goal is to detail the most efficient
way way to create an ActiveRecord migration to support a new counter cache
column.
Now that you’ve clicked on those examples, please erase them from your
memory. Iterating over the entire table, loading each record into ruby-space
(without batching, mind you), and relying on either update_counters or
reset_counters in your migration is a sure way for your next deployment to
take minutes to finish. It doesn’t take millions of records to get hit with
this pain either.
The Example
Let’s assume that we have some stereotypical tables named posts and
comments. Let’s also assume that we decided to add a comments_count
counter cache column to the posts table.
The Migration
Given this example, your migration should look something like this:
classAddCommentsCountToPosts<ActiveRecord::Migrationdefchangechange_table:postsdo|t|t.integer:comments_count,default: 0endreversibledo|dir|dir.up{data}endenddefdataexecute<<-SQL.squishUPDATEpostsSETcomments_count=(SELECTcount(1)
FROM comments
WHERE comments.post_id = posts.id)
SQLendend
The Results
With over to 25,000 posts and 100,000 comments, using SQL will take seconds
instead of minutes.
-- execute("UPDATE posts SET comments_count = (SELECT count(1) FROM comments WHERE comments.post_id = posts.id)")
-> 1.3197s
-> 26900 rows
Let’s compare that with how long it would have taken if we used a
Post.reset_counters approach:
I received a strange piece of hardware in the mail the other
day. American Express is piggy-backing on LegalZoom document deliveries with
an interesting piece of marketing – a USB key-like device in the shape of a
credit card.
Thinking it was a USB drive and one that I might be able to format and reuse
for something else, I plugged it in to my Mac. To my surprise, it immediately
opened Safari and a website pointing to an American Express credit card
offer. Huh? I thought OS X didn’t have Autorun support. So, of course, I tried
it again. This time, I noticed that the address bar in Safari scrolled as if
someone was typing really fast.
Holy trickery, Batman. It’s mimicking a USB keyboard!
NOTE: If Safari isn’t running or pinned to your Dock, the entire process
fails. They probably would have been better served to start with
⌘-space and a Spotlight search instead.
I wondered if I could reprogram this American Express USB device to behave in a
similar way as the Teensy USB Microcontroller
that Samy used.
Find the USB Device
First, let’s see what OS X thinks about this USB device. We’ll do that by
inspecting all USB devices both before and after plugging in the USB key. The
difference will help us find the device itself.
33a34,44
> WEBKEY:
>
> Product ID: 0x6662
> Vendor ID: 0x05ac (Apple Inc.)
> Version: 8.15
> Speed: Up to 12 Mb/sec
> Manufacturer: TP6662
> Location ID: 0x14500000 / 27
> Current Available (mA): 500
> Current Required (mA): 100
Note the Product ID and Vendor ID values. We’ll need those later.
I find it fairly suspicious that it mimics hardware from “Apple Inc.” Kinda sketchy.
Now What?
At this point, I was immediately stuck, but after some research, I found a
superuser question on Stack Exchange titled
Can I “reprogram” an American Express USB drive?
No direct answers, but this answer gave me hope:
It consists of a Cypress PSoC controller, combined with a 24c02 serial
eeprom … With a simple program called PonyProg you can read and modify the
contents of the eeprom.http://superuser.com/a/216810/8424
I spun my wheels for a long while with no luck, but later, I found an article
titled
Hacking USB Webkeys. This
looked promising.
Hardware Surgery (and Failure)
I slowly pried the chip from it’s plastic enclosure. It was attached with some glue.
Sweet. Contact points that I should be able to use to access the EEPROM.
All seemed well, until I tested the webkey again. I must have damaged
something, because it now skips every 5th to 8th character as it emulates a
keyboard. Shoot. Maybe next time.
Let’s [Pretend to] Read all the EEPROM
Disclaimer: I wasn’t able to test any of this, because I damaged my
hardware, but I encourage others to carry on where I left off and let me
know if you have any success!
As it turns out, PonyProg (the program
mentioned on the Stack Exchange post) doesn’t work on OS X, but I found a
program called ch341eeprom that
claimed to perform a similar task.
It also depends on libusb, so let’s install that using
Homebrew and then clone the repository so we can compile
ch341eeprom.
brew install libusb
git clone https://github.com/commandtab/ch341eeprom.git
cd ch341eeprom
Compile, and try to read the eeprom:
make
ch341eeprom -v -s 24c02 -r ~/eeprom.bin
Conclusion
While my attempt at hacking this hardware resulted in catastrophic failure,
don’t let that stop you from trying something new or breaking down an unknown
piece of tech. I learned a lot during this little investigative project, and if I find
another free marketing webkey in the mail, I just might pick-up where I left
off.
This is something that I’ve wanted to do for a long time, and I was finally
able to get my site redesigned and relaunched this week. The old design made
it 5 years.
New, Responsive, and Light:
Old, Rigid, and Dark:
The old design served me well, but it had a few problems. For one, it didn’t
work well on small mobile browsers. Now, it’s much more responsive to all
browser sizes. I also used to get complaints about my big headshot in the
bottom right corner. While that self-promotion-marketing-hack helped people
recognize me at conferences and events, I think it freaked out some of the
more scopophobic readers amongst
us. My headshot-hack remains, but I now hide away when you start scrolling. For
what it’s worth, my 3-year-old daughter thinks it’s fun to play peek-a-boo
with it, so there’s that.
I think the new site design still needs some extra work, but it’s good enough
to launch and start gathering feedback for now.
What Changed?
This was basically a total rewrite. A lot has changed, but here are some of
the key updates:
Responsiveness: Better views, font-sizes, and readability across all
browser sizes.
No More Comments: Yup, I dropped all my disqus
comments. This might upset some, but it is going to make my life just a
little bit better. Instead, I’ve enabled Twitter’s
Web Intents at the bottom of each
page. You can now reply to me via Twitter. For old posts, I did my best to
find my original tweet about the post. For new posts, I’ll do my best to
keep everything synchronized.
Jekyll to Middleman: The original site was statically generated by
Jekyll. Jekyll 2 made a lot of great improvements,
and I’m sure Jekyll 3 will continue that trend, but
Middleman has just always felt so much cleaner
to me; it’s organized in a way that I like to work. Plus, it’s easier for me
to standardize on one static site generator across the sites that I
maintain, and I
honed in on Middleman
well before Jekyll v2 launched.
Twitter Card and Open Graph Support: You know those nice catchy pictures
you get when you share a website on social media? Yeah, I got that. Learn
more about Twitter Cards and
Open Graph.
What’s Next?
I have a lot of little plans for this site, including doing a better job of
promoting the things that I work on day-to-day, but in general, I hope the
redesign also rejuvenates my desire to publish new relevant content about
programming, the web, business, entrepreneurship, tech, etc. No promises, but
that’s the goal.
BOULDER, CO — Boulder Ruby — I had the opportunity to present at Boulder Ruby during Boulder Startup Week 2015. I covered several online services that not only have excellent Ruby support, but also can help accelerate your MVP and/or product launch. The finale included a live demo of Cloudinary, Pusher, and Embed.ly services.
I was recently frustrated with the documentation on the HTML5 autocomplete
attributes and Chrome’s response with the x-autocompletetype vendor specific
attribute.
These attributes are typically used on <input> elements to better signal to
the browser the best way to “autofill” a form. The problem, however, is that
there is no official standard yet, and there are mild differences between
values used for autocomplete and values used for x-autocompletetype.
ST PETERSBURG, FL — 1 Million Cups - STP — I had the chance to tell a bit of the story behind BusyConf while also gleaning some advice from fellow entrepreneurs.
In this episode, we talk with Ryan McGeary CEO & Founder of BusyConf about conference registration, mobile event apps, HTML vs. native event apps and more.
I use GitHub Pages for hosting some of my
websites, and I use the Middleman static site
generator as my content management system for some of these sites.
I like to run continuous integration for my projects whenever possible, and
this goes for static site repositories as much as regular code repositories.
Recently, I started playing with
Codeship. It’s
a well-priced continuous integration and deployment service, and I wanted to
be able to automatically deploy my middleman site upon successful builds to
the master branch. It took a bit of trial and error, but I finally got
something that works well.
Test Settings
After initializing a new GitHub repository in Codeship, you’ll need to define
how tests are run. Under the Test project settings in Codeship, select
Ruby as the language and define your Setup Commands and Test Commands.
Setup Commands:
bundle install
Test Commands:
bundle exec middleman build
Perfect. Now your site will run a test build upon every code push.
Deployment Settings
This is where I had some trouble, but I finally found a set of commands that
correctly deloyed to our gh-pages branch in the repository. First, it’s
worth mentioning that I use the
middleman-gh-pages gem for
deploying to the gh-pages branch. This gem gives you a rake publish task
that handles most of the dirty work.
Under the Deployment project settings in Codeship, configure a deployment
from the master branch using a Custom Script.
This configures a git user, removes the build directory that was left there
from the test steps, adds a gh-pages remote branch (because Codeship only
clones the relevant branch during setup), and runs the rake publish task to
deploy the site to the gh-pages branch.
If all went well, your site will automatically deploy to GitHub Pages upon a
push and successful build in the master branch.
GitHub Publishing Caveat
Make sure the email address that you use for this git user is a verified email
address in your GitHub account. Otherwise, GitHub will accept the commit, but
it will not publish the changes to your site. To fully make everything work,
you will need to do one of two things: 1) Move the Codeship deploy key to an
SSH key in your own GitHub account OR 2) Create a new machine user on GitHub
and move the Codeship deploy key to an SSH key on that user account, give that
machine user push/pull access to the repository, and make sure that machine
user’s email address is fully verified by GitHub. I highly recommend going
with option #2.
Extra Credit: Force Codeship to skip builds on the gh-pages branch.
Unfortunately, Codeship currently tries to run builds on all branches,
including the gh-pages branch. This is undesirable for this setup, so to
avoid this, we also need to add “–skip-ci” or “[skip ci]” to the commit
message that is pushed to the gh-pages branch.
Fortunately, after
this pull-request by
yours truly, middleman-gh-pages can support that.
If using version >= 0.0.3 (or the master branch), you can add this to the
bottom of your project’s Rakefile:
# Ensure builds are skipped when pushing to the gh-pages branch
ENV["COMMIT_MESSAGE_SUFFIX"]="[skip ci]"
LEESBERG, VA — TEDxBalchDriveWomen 2013 — Software — “Luck” is too often attributed to one’s success. Let’s cut the modesty and tell better stories about the good and bad decisions that got us where we are. No one can learn from your success if you merely attribute it to “luck.”
Disclaimer: I was asked to give this talk at the very last minute. I’m happy with the message, but I’m a little disappointed in the delivery. I wish I had more time to prep for my first TEDx presentation.