The Engineering Assistant

I made a snarky comment in our chat channel the other week about having an executive assistant concept for engineers and what was initially a joke became an interesting idea to me. I haven’t researched it so it’s probably already been talked about but my thoughts on it are outlined below.

Executives have long had the concept of an EA or Executive Assistant. A big part of their job is to offload some of the non-critical tasks that need to be done, but don’t warrant the hourly rate or responsibilities of a CEO, CTO or VP. For example, an EA will help with scheduling meetings and working around attendees schedules, organizing logistics for events, and triage correspondence like email, etc… you get the point. A CEO’s role is to drive the company forward and provide vision and leadership; anything that takes away from that focus should be offloaded. If your CEO is looking up people’s availability for a meeting and spends 10 minutes on that task, the company is worse off. I believe this same concept can apply to Engineering teams. Companies spend huge sums of money optimizing manufacturing processes but most spend relatively fewer resources on optimizing engineering productivity.

I’ve been in software for 20 years across a number of companies and organizations from small startups to large enterprises. They all have a similar technical structure. You have tiers of technical abilities. You have an upper tier with the technical visionaries, architects (hopefully not ivory tower) and principal engineers. Then you have senior engineers, mid level and jr developers or interns. Engineers in the uppermost tier tend to have a multiplying effect in the organization.  A big part of their role is to provide technical leadership, drive the vision at a technical level, mentor and tackle some of the harder problems in the org. A lot of times this group is paving the way for the core IP of the company. They also try to look ahead and make sure the company is in a good spot for the future.

Some days are spent writing code, software abstractions,  debugging many of the critical issues that come up. There are also weeks when they have to perform due diligence on a potential acquisition, or do detailed evaluations of potential new partners and/or vendors. They may be frequently asked to de-brief C-level execs on architecture, future planning, or potential product features.

Like a CEO, their time is sparse and valuable. In any typical day for these individuals there are pockets of time spent on mundane tasks that could be offloaded. Unlike an internship where someone is temporary and perhaps one day full time, the new EA (engineering assistant from now on) is a full time hire that would be embedded with their engineer. When your engineering teams are critical to your business you want your best and brightest minds focused intensely on solving the hard problems, technical challenges and communication around technical directions with the teams. Just as a CEO, spending 10 minutes on scheduling a meeting and finding free slots for 20 people isn’t a good use of their training either. This isn’t about making time for people to play more golf, it’s about matching roles with problems.

Line Fitting

As an example below, we have an engineer with a blue line representing the expected impact they have in the organization. The first graph shows tasks as red dots. We see we’re fitting that line pretty well. We have some tasks that make a big impact and tasks that fall just short or about as expected with that person’s performance but generally it’s a well fitting line.

Screenshot 2016-04-17 10.53.29

In the graph below we see an engineer working on tasks that are well below their expected impact levels. We have a mismatch of expectation and reality. We need to make sure we’re fitting our lines properly.

Screenshot 2016-04-17 10.54.09

“The last source of major competitive advantage is the selection and placement of people” – Drucker

Just as startup CEOs don’t usually have an EA, the Engineering EA would typically apply to mid-size and large organizations where differing geography and increased number of internal teams teams are likely.  As a company gets larger and more spread out, there’s more connection points that require better communication. It’s all about opportunity cost. Did your company miss out on shipping a major release because your top engineers spent 10% of their time on non-critical tasks? If you had paid an EA $50,000 a year, could that EA’s engineer have shipped faster with higher quality and made the company $3 million from hitting the marketing deadline? That’s what I mean by opportunity cost, and it happens all the time in software.

When I look back at previous jobs and I try and think about the people I’ve worked with in those roles, I started to compile a list of some of the things that having an EA would have helped with. When I think of an EA I think it’s also a great opportunity to unlock potential talent. Most engineers in software nowadays have to have comp sci degrees and training in data structures, algorithms, etc.. You could hire an EA with a much lower barrier to entry and cost. A smart, hard working person with a desire to break into the field but doesn’t yet have the experience or education could be a great fit for this role.  Someone that could be trained and mentored by the orgs best talent. I’ve seen a quite a few folks like that shine in organizations in the past. You would build a relationship with your EA and they would eventually learn how you like to work and the relationship becomes more efficient over time when things can be predictably achieved. A true test of success would be an EA transitioning to an engineering role within the organization and in turn spreading the great lessons they were taught.

Here are some example tasks that are part of many more senior engineers’ daily lives. Some of these are more realistic depending on how the devops/dev owns qa interactions are in your company.

  • Meetings
    • Jennifer is having a meeting on Friday about our new “foo” that I’m really interested in. I’m pretty swamped getting this release out. Could you sit in that meeting, take notes and message me if there’s anything I should be in the loop about.
    • Can you schedule a meeting with our database vendor when it works for both our schedules? I need more details around that issue we hit last week. Make sure to invite Aaron from the site reliability team as well.
    • I need to fly to the Calgary office for a vendor meeting. While I’m there, I should probably do some architecture reviews with the teams there. Can you schedule  reviews for the 2nd and 3rd day with the Foo and Bar teams? If that works then msg me over a copy of the design docs from their latest revision that I can review on the flight.
  • Communication
    • We settled on a new abstraction for accessing our datastores. I want to give a quick tech talk to the team and show them where to find all the docs and answer their questions. Can you set up a town hall with all the engineers two weeks from now? Make sure we have the conference room booked so the local folks can be in the same room. Have that place that everyone loves cater in some food as well.
  • Email
    • Could you triage my email in the mornings. If anything from support, production notices or questions from executive staff come up ping me right away, otherwise triage and label any anything else and schedule something on my calendar to make sure I respond to those items.
    • We’re really in crunch time, can you monitor my email and chat messages. I’m going dark for two days with the team to help the team ship this feature for our next release. Ping me if any emergencies come up.
  • New Application / big change released
    • I just released that new application into production, can you monitor the logs first thing in the morning for the next week and report any  anomalies to me while I build out the first version of the monitoring dashboard.
    • Can you build out a dashboard for the logs from that new application and set up the alerts to tie into our alerting systems. I want to also be paged any time database latency goes over 100ms for the next 2 weeks.
    • Can you set up the monitoring dashboards for the new App, I labeled all the meters I want in there with the suffix KPI.
    • While I’m setting up the test automation, can you run through the app for the next week and do these 5 things, then check the logs and metrics and make sure everything is still running smooth.
    • Can you work with the QA team and get external monitoring setup for these 6 HTTP endpoints for this new service I just launched.
    • I wrote a few basic integration tests so far, can you finish up that project and make sure we get better coverage on how foo interacts with bar?
  • Support issues
    • I just got a call from the support team. One of our large customers is complaining about the App the team just released and they need me to help triage. Can you do me a favor and grab the logs of some sample hosts from our logging system while I get my test environment set up? Filter out any of the info statements for me as a first pass.
    • I just got a call from the support team. One of our large customers is complaining about latency in the App the team just released can you check out the metrics dashboard for the past 2 weeks and see if there are any spikes on latencies anywhere, then ping ops to see if there were any network related issues in the past 2 weeks we should be aware of.
  • Pre-Deployment
    • I need to have a meeting with our operations team on the requirements for this new role in production. Can you put together the list of services we need to talk to, the environment variables and settings that need to be made, then schedule a call with the ops team.
    • We’re going to need a new role for this project. Can you set up the role template, get all the directories and setup in place. Work with ops or our internal tools and get a running server up in the dev environment we can start iterating on.
  • Simple scripts / automation as they progress in skills….
    • Can you write a quick command line script that does A, B and C then get that out to the team? I think that will save everyone a few minutes of their day.
    • Sarah sent me the new ElasticSearch schema for users. Can you do a quick once over and make sure it matches up with our data model in the code? Let me know if there’s any differences in structure. I’m planning to look at it tomorrow but it will save some time if you check for low hanging fruit.


As you can see those tasks above are usually part of everyday life but you’re not paying those people some of the highest salaries in the company to focus on those things. Having an EA that can be trained and mentored with an engineer can be a valuable competitive advantage and you need to look to optimize your organization at all levels.

So what do you think? Crazy idea? Worth exploring?  Hit me up on Twitter and let’s talk about it

Moving a team from Scala to Golang

Scala has long been part of the CrowdStrike stack, the primary language in fact. I helped lead the adoption of Scala as we first started to develop our applications back in 2012. In fact it was one of pros for my decision making process of wanting to come to CrowdStrike. Several of the early developers were interested in adopting it as well so it seemed to be a nice fit.

I had come from a company called Gravity, who were also heavy Scala users. It was the primary language there. I was used to it, enjoyed it, saw the power of it and thought I could prevent some of the issues I saw with Scala as CrowdStrike grew. We were doing high scale analytics, batch jobs over Hadoop and our Chief Architect (hi Bissel!) was doing lambda architecture before it was what the cool kids were doing.

A recent quote from one of our senior engineers prompted me to finally write this post describing why we’re transitioning most of our stack to Go and why new services default to Go by developer choice.

Instead of waiting until the end of the this post I should clarify that Scala will not be leaving our stack completely. In fact it will complement where Go does not shine. Scala is a big part of our machine learning / analytics stack. It’s interop with java projects we use, and its ability to provide a nice DSL that our analysts can use still make Scala a solid choice. It’s becoming more of a specialized tool vs the core development language.

I’m going to take you through this from the lens of a Technical Director. A lens where you need to scale a company from the early days of 5 engineers to 200+ engineers as the business grows. It’s about having a maintainable code base where you can have people cross projects easily and get new hires up to speed rapidly.

I remember when I first saw the potential issues of scaling Scala at Gravity back in 2009/10ish. It was close to the end of the day when we had a major issue reported in production that was affecting our large customers. Several of us started investigating and were able to track the source of the issue. The only problem was we had no idea what the code was doing at first. We came across a strange symbol we hadn’t seen in our projects before. The spaceship operator <|*|> . Someone said out loud “what the hell is that?”.  There was some other implicit magic going on that wasn’t immediately apparent. A CMD+B to traverse into the method yielded nothing in our IDE as it couldn’t find the symbol (IDE’s have improved here since). A quick googling of “<|*|>” yielded nothing as well. We were stumped and didn’t have sources pulled down. [1]

Screenshot 2015-11-21 11.20.00

The developer who wrote the code was unreachable on vacation so we had to figure it out. We noticed a new library was being included called scalaz, hours later we had tracked down the mystery symbol and grok’d what was going on, made the fix and life was good again. That blip turned a fix that should have taken minutes into a fix that took hours. That was the point I started seeing the split in our engineering team.

Scala is a powerful language, it comes from academic roots and gives enough flexibility that you can easily start writing “write-once” type code.  Scala developers typically travel down two paths: You have the “it’s a better java ” camp you have the “I (heart) Applicative Functors” camp.

Screenshot 2015-11-21 12.07.04

The “it’s a better java” camp like the terseness of Scala, and  the standard features that make Scala generally more enjoyable than Java. There’s functional programming in their new Scala code but it’s not the main focus. The “I (heart) Applicative Functors” camp really takes to their new functional world and begin to expand their knowledge deeper down that path or they bring their already functional backgrounds from places like Haskell.

Based on my experiences you start to split down these camps and have excellent programmers in each so you can’t say one side is superior to the other. On the semi-functional side you potentially have more generalists working across languages, or those who may not want to learn lambda calculus theories to work with an API server.

As an example, this is a code sample from a project we had one of our more advanced Scala developers start:

Some of you may look at that and say awesome, but some will say WTF is that? There were thousands more lines like above. This was to be something the whole team could work on but half the team didn’t want anything to do with it. The developer who wrote it is a brilliant person but the fact it divided half the team was a problem. Luckily this was caught in code review and rejected based on our internal guidelines. It never made it out to production.

As you’re scaling an engineering team this split becomes more apparent when trying to get new hires up to speed. Scala has a lot of rough edges already around getting a build environment, SBT pain, IDE environment pain , release upgrade pain, slow build times, add on top of that a heavy dose of functional concepts required for proficiency and the ramp up time grows and dev output slows down. Now there’s more upfront training from existing developers required which slows things down as well. SBT is also a real sore spot. There’s always that one person who actually knows what the heck SBT is doing and can debug everyone’s issues. I’m aware that Scala is not scalaz and they can be mutually exclusive, but I’ve seen issues with or without it.

It’s also not a question of Scala being “too hard”. I’ve never had someone not be able to learn the language. It’s about the investment. You make an investment in something with the hope of it paying off somehow down the line. Whether that’s faster time to market, higher performance/lower cost,  or increased stability. There are specialized places I’ve seen that happen with Scala but not in the general case. We discussed as we were about to scale up if we wanted to invest in more docs, guidelines, example projects, etc… but the reality was we didn’t think that investment would come back vs Go.

This isn’t unique to companies I’ve worked for. Twitter has gone through the same growing pains, as well as other companies I’ve talked to at conferences and people I know working with Scala. It’s a common theme, in fact. While you can have very high performing small teams going with Scala, trying to grow and engineering organization > 50 is an uphill battle. If you’re already invested in the JVM, Java8 is a solid choice that borrows some of the concepts of Scala to make Java easier to work with.

Other references from larger teams seeing issues:


Is LinkedIn Moving off of Scala?

Former Twitter Platform VP heard saying Java8 may have been a better choice if available when they made the choice of Scala for similar reasons

You can also see a trend at Twitter where the latest OSS projects released are in Java (Heron, DistributedLog, etc..) In fact a telling line in the Hero release reads: “It is written in industry-standard languages (Java/C++/Python) for efficiency, maintainability, and easier community adoption.

Is Scala on it’s way out? (article):


and a funny Tweet about the subject:

Screenshot 2016-02-17 07.52.50

That’s where Go enters the picture. One of Go’s reasons for existence is to make developers more productive, limit the number of ways you can do something and have a very opinionated view of the world at the compiler level. I pushed back on Go adoption internally for a while. I worried about splintering even more, having another language for people to learn as we already had quite a few technologies in play. We have an internal policy about looking at adopting new technologies when you have at least 3 people willing to support it at 3am if there’s a production issue. After much prod’ing (thanks Sean Berry) and getting past that 3 number, I dug in to the Go world and saw it solved a lot of issues I had with Scala at the organizational scaling level.

Fast build times, small binaries, one file, built in formatting, great tooling, built in test framework, race detector, visual profilers, a nice concurrency model? Wow, sold! We did a sample project in Go that was successful, then another, then another, expanded out the number of developers we had on Go and it started to become the language of choice people wanted to write in. You can jump into any Go project and know immediately what it’s doing. Do I miss immutable types and some of the great features of Scala? Sure do, but I think the maintainability side of the story is too great to overlook with Go. We’ve seen faster ship times, better stability and better test coverage being written.

One of the other benefits of Go was widening the pool of backgrounds we can hire. We can take someone from any language background and have them ramped up on Go in weeks. With the Scala side there’s the JVM learning curve, the Java world of containers, black magic of JVM tuning, profiling tools,  etc…

New developers we’ve hired are ramped up in weeks vs months (we have lots of services that operate at extremely high scale, across several divisions).  We now have the majority of our services written in Go and one of the last holdouts to move to Go just wrote his first project and said to me afterward “Wow, I read through that library once and I knew exactly what it was doing , I’ve read the Scala version of that library four times and I still have no idea what it does, I can see why you guys like it so much”. That was one of our senior engineers who’s previously worked for one of the largest web properties in the world. This process was a complete bottoms up initiative from our development team who pushed for the move to Go.

We now process hundreds of thousands of messages per second and Terabytes of data per day with our GoLang services. Some try to equate Go’s simplicity with weakness, I’ve seen the opposite. You can do some pretty powerful stuff in Go. There is power in simplicity. The error handling, while seemingly annoying at first, actually has lead to more robust error handling and stability in application code. You can’t just throw something and hope it gets caught somewhere.

Google is also a major investor and having that technical alignment opens up access to additional resources that we can leverage.

I’m not here to bash Scala, or ScalaZ (I love ValidationNel! ) but more to give some real world context of Scala in a production environment over 7 years with two companies. I still use Scala and love hacking in Scalding.  Some of our more ambitious projects coming up will most likely be Scala based but I’m just not as sold on it anymore as the core language when trying to scale a fast growing engineering team. There are always exceptions and if your team really loves Scala you can make it work, and some companies are.

Go sits in a place that happens often… when you need small’ish, high performance services. Where you’re doing light transformations, shuffling data around and putting APIs in front of data or supporting systems.

Go just makes it too easy.


If you’re interested in hearing more or chatting, follow me on Twitter

…and we’re hiring 🙂



** Marius, one of the twitter Scala gurus, also seems to be getting keen on Go.

screenshot-2016-09-24-08-58-56 screenshot-2016-09-24-08-58-42





[1] Code Reviews would have solved this issue as someone else would have stumbled on that symbol and asked for more clarity and understanding. That would have help introduced the library to the team easier than a surprise. Unfortunately, at Gravity we didn’t have required code reviews in place. We do have code reviews in place at CrowdStrike. However, given not every person is on every code review there’s no guarantees.




Path to Productivity: No Meeting Thursday

At small startups there’s usually very few meetings initially and the number of people you have to communicate with on a daily basis is typically quite manageable. You feel productive and you’re cranking out code, designs or other tangible deliverables. Life is good! Your social graph at work looks similar to the graph below.


Hopefully, your company is successful and you start hiring. As you hire, your social graph at work also grows. Perhaps not on every hire but as the company expands the number of people you need to interact with will go up as well. As you get even bigger your social graph looks like the new graph below. Now things aren’t as fast. You need more co-ordination, more people have to understand what’s going on. You start to have more meetings, and if you’re distributed, potentially even more.



You start to become sad that you’re not as productive as you once were but understand it’s part of growing. Then you get to a point where you feel like you’re in too many meetings, they’re scattered throughout the day, you’re working in bits and pieces but can’t really tackle anything of substance. You need a long stretch of time for understanding a problem, building solutions and algorithms in your head. You find yourself having to work after-hours or early in the day just to be able to chew on meaty projects and feel somewhat productive again. If you’re in a big company this process can happen even faster.

More than half the engineers I interview cite “too many meetings” and “not feeling productive” as one of the key drivers as to why they’re looking to leave their current company.

After hearing this at my previous company during interviews I wanted to instill a guard against that early on at CrowdStrike. Very early on we put in place the concept of “No Meeting Thursday”.  Meetings were not a huge problem early in our life, our social graph was small. However, I knew as we grew and expanded there would be more interrupts throughout the day for engineers. I also know if you bake things like that into the culture early on, it’s easier to fight to keep it, than to fight to instill it.

So with that, I put a calendar invite on everyone’s calendar that blocked off the whole day for No Meeting Thursday. The goal of No Meeting Thursday is not to have a “don’t talk to me” day. It’s about not having anything scheduled or re-occurring to give engineers (and other groups) the chance to own their own schedule. You’re an adult, do the right thing.  If they need to be alone for 8 or more hours to really dig into a problem, great that’s what the day is for. If someone is working on a team and that team wants a whole day of uninterrupted time to focus on design, testing or anything else related to their project and everyone is on board then great, that’s what the day is for. Obviously, production issues still come first.

As new executives come in there is a training period of “What’s this No Meeting Thursday thing about and why did you deny my meeting invite.”  I wrote this post to reduce the number of times I have to explain the concept so here it is….

Managers and Engineers are usually on different schedules. Paul Graham famously calls it “Manager vs Maker schedule“.  Managers are used to being interrupted and work on broken chunks of the day. They work in blocks of time and usually have calendars that will make you cry. Managers can go from meeting to meeting with minimal context switching. Engineers on the other hand work in long stretches and need hours to get into the zone where they’ve built a problem in their head enough to the point then can work directly out of mental RAM. That’s where the productivity sweet spot is.

I like to think of meetings this way for engineers… You’re an engineer, you’re building  a mental chandelier in a big, tall entryway. You have a high ladder where you climbed to the top and you’re starting to put together your big, beautiful chandelier in your head. You added the base, you start adding crystals, things are going well. You’ve got a great mental model of your problem and some corner cases.



Then a calendar reminder pops up, reminding you that you have a meeting in the middle of the day. You head to the meeting to talk about something completely off topic and Boom!!, Crash!! Your mental chandelier starts to wobble and fall, crashing on the floor. Not all the crystals are busted but you definitely are starting to swap out that mental RAM for this new topic in the meeting. Meanwhile you’re actually still thinking about where you left off in  your other task so you’re only somewhat paying attention to this new topic. So you’re losing context on your previous project and this new subject is getting half your attention.


The meeting ends and you have to clean up to start rebuilding your chandelier. Hrm, where was I?  You may get back to your desk and you think you’re already interrupted so why not just check email real quick. You read an email and start to think about that and now you’re two levels removed from your actual project. Ok time to refocus… This could take anywhere from minutes to hours. If it’s towards the end of the day you might just say forget, you’ll start again tomorrow and work on little tasks the remainder of the day. I tried to find the source for where I read that chandelier analogy before but couldn’t find it. If someone comes across the source feel free to drop a comment so I can attribute it properly.

As someone who’s been in both the management and and engineering side I can validate for myself and many others I’ve talked to this is the way it goes. When I’m in a managers schedule I know I’ll be interrupted all day so things involving deep thought are left for early in the morning, or after office hours. In my opinion a good manager needs to understand the affect random meetings have on engineers and the time wasted going in and out of context.

The phrase of the day nowadays is “more productivity”. To put it as bluntly as possible. Would you want to work at a company that couldn’t even give you 8 hours of uninterrupted work time? How can someone be productive if you never give them long stretches of time? It sounds silly when you say it out loud but that’s what a lot of companies do by default. With so many connections people could be interrupting each other left and right.

Scheduling random meetings in the afternoons? The worst. If you are going to schedule meetings for engineering teams, keep them in blocks. If I’m already interrupted at least keep it going so I can get a long stretch of time in the afternoon or in the morning. Good managers keep tabs on meeting workload and fight to reduce it. They take meetings for the team and distribute information asynchronously or ask that meetings be changed to discussions in a chat tool, or via a ticket as done in the open source community.

I’ve heard of other companies doing no meeting afternoons, however, for distributed teams in many time zones that makes it hard to establish when that time will be. Having a full day evens it out more across the globe.

Meetings can be fruitful but can also be highly disruptive. For engineering teams please be conscious of the effect meetings are having. Because if you’re not, you may find yourself having to hire replacements for a situation that could have been avoided.

We’re 3 years into No Meeting Thursday. I put out an email to the team recently on if we should preserve NMT and if people thought it was still effective. The responses were an overwhelming “please keep NMT, fight for it and let me work”.


If a culture like that interests you, ping me… we’re hiring 🙂

or follow me on Twitter:


thanks for the reviews from:

John Kurkowski

Roger Clermont

Your guide to the 2014 Amazon AWS re:Invent Conference

I’ll be attending my 3rd Amazon AWS re:Invent conference this year (as a speaker this time!) so I thought I’d put together some of the tips and tricks I’ve found over the last two years.

Screenshot 2014-11-08 14.23.34

I’m going to take the view of the advanced AWS user. If you’re new to AWS congrats, everything will be new and great. If not then you’ll want to think about some of the points listed below. Overall re:Invent is a fantastic conference and it’s huge.  Amazon spares no expense. Rumor is they put up 6 figures to the Venetian to bump up their wifi capabilities in the first year to support a growing number of techies attending. The parties are pretty awesome and everyone on staff is amazingly helpful. The first year was great but sometimes painful… over packed rooms and all the good talks were mostly unavailable. The netflix talks had lines around the casinos and you had to wait in line a full hour earlier and miss a talk to get into some of the advanced sessions. bleh. Last year they took notice and expanded all of the rooms and I had no issues with waiting or missing a talk I really wanted to attend. I have no reason to believe this year will be different, should be plenty of space for all.

(Photo of one of the lines for a Cockcroft talk the first year)2012-11-28 16.16.03


One thing I’m not super excited about is the sheer number of simultaneous talks. At one point there are 19 simultaneous talks going on. That’s way too many IMO. You usually want to hit 2 or 3 that are happening at the same time  so it’s sort of a bummer. They should have fewer, higher caliber talks. Last year there was a good deal of “salesy” type talks where after 10 minutes you’re asking yourself why you just skipped that other talk you wanted to attend.  Generally avoid any 100 or 200 level talk from an enterprise vendor or large company. Tweets on those last year were pretty bad, more sales pitch than useful information, and I walked out of a couple myself to catch the rest of another talk. Example: “We utilized AWS to streamline our customer product delivery system, improve our response times and increase our sales output by 30% , we won’t tell you how, we’ll just boast about high level shit we did and how you can integrate with us now” :/


If you’re there to learn some focused information attend as many 400 tracks as you can. Amazon actually puts on some good advanced topics such as the internals of EMR map/reduce, EBS performance deep dives, etc… If you’re looking to learn, that’ll get it done usually. There aren’t a lot of them, they usually sell out and are packed. Get there very early so you’re not sitting on the floor.


Go New! Go to talks you know nothing about but may have heard or may have some interest in. If you already know the ins and outs of AWS then most talks will leave you wanting for more. You may walk away with one or two nuggets but with only 45 minutes, 20 mins of that is usually intro/setup and 25 mins for meat. That doesn’t leave a lot of room for deep inspection of topics. Instead let that 45 minutes be completely new. You’ll get exposed to more things and actually learn more in that 45 minutes. For the topics you have experience in, wait for the videos to come out to see if you missed anything. When in doubt, flip a dice (although with so many tracks you’ll need 3 dice)

Tip 3

Power!!! you’ll eventually need power to charge your laptop and phone. In a 2,000 seat room, the only power plugs are usually against just one of the walls, since most walls are moveable. You’ll want to get to a talk early to grab a power seat. They go fast. Better pro tip would be to bring a power strip with you so if it’s full you can unplug everyone, plug in your power strip, reconnect everyone and add yourself. You might even get a thank you that someone else can now plugin next to you 🙂

Tip 4

This annoys some people but is the only way you’re going to get to the next talk in time to get that sweet power… sit in the outside seats in your row. If you’re in the middle, after a talk it’s about 10 mins to empty out the talk so you’ll be waiting for some turtles ahead of you. If you’re on the outside you can break out right as the talk ends and get a nice spot in the next talk. Be somewhat courteous and make yourself small so people can get by easier.

Tip 5

Ready for dining? Public House  is the spot where the cloud gods hangout. Here you’ll find your Cockcroft, Vogels, and Berkholz’s of the cloud world. It’s a great spot for beers and people watching. You’ll get to interact with other techies and talk shop over some great craft brews. For breakfast, the best pancakes I’ve had in my life are at the Grand Lux Cafe . That’s where you’ll find me every morning 🙂 I usually gain a couple pounds just from breakfast but it’s worth it.

Tip 6

Get outside! The first year of the conference I realized on day 3 I hadn’t seen the outside world in 3 days straight. The Venetian has everything you need and more. Kind of scary actually… make it a point to stretch the legs and get some sun on your face.

Tip 7

Go to the parties! AWS spares no expense with their parties either. Deadmaus was pretty awesome last year, you could walk right up to the stage. They had laser limbo, mini helicopter landing games, old school arcade games and more. It was a good time, throw your anti-social caution to the wind and just go hang out for an hour.

2013-11-14 22.31.40

Tip 8

The shirts! I still wear my re:Invent shirt from 3 years ago. They last forever and are high quality. I wear my re:Invent shirts every week still. Make sure to get yours at the party.

Tip 9

Go to plenty of Netflix talks. They are some of the high scale AWS pioneers and have paved the way for you. Hear some of their problems/solutions and it will help you with your architecture. They also have valuable content. All of their talks are good, they’re all about the bass.

Tip 10

Vendor booths…. Lots of vendors to see and talk to at the conference. They all give shirts away so get there early if you want swag, they run out pretty fast. Most vendors were pretty knowledgeable and the companies sent their best reps out for this big conference. Be forewarned, if you let vendors scan your badge for a shirt you will be spammed for weeks afterwards. Fair trade I think.

Tip 11

Use twitter heavily. Follow the hashtag #reinvent to keep up to date on good talks, people trying to get together to discuss topics, and locations of beer meet-ups. If you’re looking to network that’s the best way to keep up with the pulse of the conference. I met a great engineer over twitter using the #reinvent hashtag who wanted to talk shop and have a beer at the public house. We gabbed for a few hours that night. Shout out to Martin Cozzi 🙂

Tip 12

Utilize AWS Solutions Architects. In the vendor area Amazon will have solution architects available for every service they have. Bring your problems and they will get on a whiteboard with you and get into the details. Wanna know the best Dynamo schema or write capacity heuristic? Wanna know if your EMR workflow is sound or stupid? Someone will literally sit with you and get you a solid solution. They’re there, free and they have their most senior people attending. Take advantage of that knowledge sharing.

Tip 13

Werner Vogels is cool as shit. You may not meet another C-level exec as cool as Vogels with an Amazon size market cap. He does his presentations in Nirvana shirts and converse. I was lucky enough to meet him at the party last year and he couldn’t have been more excited about the event and friendly to talk to. Don’t be afraid to go up and say hello. You might even catch a nugget of knowledge.


Me and Vogels

Wrapping up

So that’s some tips off the top of my head for attending this year. If you want to see an advanced talk Mr Sean Berry and I are going to be representing CrowdStrike and doing a talk on how we do our blue/green deployment architecture.  APP307 – Leverage the Cloud with a Blue/Green Deployment Architecture We’ll be getting into the nitty, gritty of how we utilize a blue/green deployment model at scale with terabytes of daily data ingested. Kafka, zookeeper, load balancers and more!

Follow me on twitter for frequent updates and if you want to grab a beer while you’re there. I want to have some deep tech discussions while I’m there so let’s nerd out.

If I missed any good tips, leave a comment.

Speaking at AWS re:Invent 2014 conference

Screenshot 2014-11-02 08.57.27

I was lucky enough to be selected to speak on behalf of CrowdStrike at this years Amazon Web Services re:Invent conference on the subject of blue/green deployments. If you’ve been looking to grok blue/green deployments or get to the stage past the high level “just switch your load balancer” type talks then this talk is for you. Leverage the Cloud with a Blue/Green Architecture

We will get into how blue/green works with the data plane including how to use it with a stream processing data system where Message Queues and/or Kafka are involved.


Screenshot 2014-11-02 09.00.46

I’ll be co-presenting with Sean Berry from CrowdStrike and we’ll be happy to talk tech over beers after the talk. See you in Vegas 🙂

Talk Overview:
APP307 – Leverage the Cloud with a Blue/Green Deployment Architecture

Minimizing customer impact is a key feature in successfully rolling out frequent code updates. Learn how to leverage the AWS cloud so you can minimize bug impacts, test your services in isolation with canary data, and easily roll back changes. Learn to love deployments, not fear them, with a blue/green architecture model. This talk walks you through the reasons it works for us and how we set up our AWS infrastructure, including package repositories, Elastic Load Balancing load balancers, Auto Scaling groups, internal tools, and more to help orchestrate the process. Learn to view thousands of servers as resources at your command to help improve your engineering environment, take bigger risks, and not spend weekends firefighting bad deployments.

© 2022 Jim Plush: Blog

Theme by Anders NorenUp ↑