March 5, 2018 Why Crowdsourcing is a Full Stack Developer's Lifeline

When I started my software career over two decades ago, delivering enterprise software was a team process. And when I say “team,” I don’t mean the type of team that works together to win the game, but more like, “This is really hard and no one can understand it all, so we need a team of people” or rather, “No one can be good at it all.” But how things have changed. Just like the slow-moving (and, I am sure, delicious-tasting) Australopithecus afarensis no longer runs from the hungry saber-toothed tiger, we have evolved into hominids that can deliver on all aspects of software delivery.
Back in the day, you first had to get the hardware people to procure a couple servers for you (at least one for the database and one for the backend). Then you needed to get the DB person to install Oracle and provision a DB for you, and maybe even set up the tables. Then an OS person to set up the drivers, a security person to patch and bless, an Apache person… you get the picture. And forget dev, testing, and prod environments — this was just for one set, and you were happy to get it in a few weeks. This single environment was “dev” until you were done. Then it became “testing,” and then it became “prod.”
Like all good developers, you were able to hit every single requirement, and the users were so happy they had no future enhancement requests. So there was no need for superfluous dev and testing environments. And then something remarkable happened.

The rise of the full stack developer: VMware’s ESX

In the early 2000s, VMware came out with their ESX (i.e., virtual machine hypervisor) and it was finally ready for primetime. In my last role at a mobile provider with Dustin Weaver (dweaver)and John Wheeler (jwheeler), I was turned on to the power of VMware’s ESX by Matt Twomey (mtwomey). After playing with it for several weeks on the Ubuntu server under my desk (hostname: countrysquire), Dustin and I approached the manager of the hardware and DB team (ironically, John) and requested the biggest server (BFS) that our budget would allow — for “dev testing.” It was the only label that would would allow us root access. Under the cloak of darkness, Dustin and I installed ESX. It was the last hardware we ever procured.
A few months later, John asked me, “Do you know why my team is not getting any work requests from “app-dev” anymore and are sitting around on Usenet or playing Doom all day? Does it has anything to do with the big testing server you guys manage that has thousands of open sockets and is consuming the bulk of the bandwidth to the data center?” Half joking, I said, “Our goal is to put you out of a job.”
This was a really profound moment in the history of application development because it allowed for the black magic around setting up an operating system, database, or application server to be easily templated and replicated every time with just a few keystrokes. Little did I know that this was the first fortuitous event that would lead to the rise of the full stack developer.

The rise of the full stack developer: open-source software and sharing

The second fortuitous event that led to the rise of the full stack developer was the acceptance of open-source software in the enterprise in the early 2000s. Together, Linux, Apache, MySQL, and PHP (i.e., the LAMP stack) became a widely accepted platform for web development because it was free and so well documented. Ironically, web searches and community forums became a more powerful means of support for these open-source platforms than a paid service contract of the closed-source rivals. Soon software product companies realized that it was easier to sell their solutions if they did not have to bundle a license like Solaris, Windows NT, Oracle, or IIS.
This brings us to the third and final fortuitous event — sharing our creations for free. As software developers, we explain how we do things in exhaustive detail. And often, the desire to share what we’ve made overshadows the desire to make money from our creations. That tiny flaw in our collective personality has allowed computing to make astronomical leaps and bounds. The fact that today’s crowning achievements become tomorrow’s cornerstones is what enables us to shop for feature-rich JavaScript libraries and implement them with the ease of pasting clip art. Npm install, pip install, Docker Compose, and git clone have become the brick-and-mortar foundation for developers who want to whip out a REST service in the morning, train a machine learning algorithm before lunch, restyle a mobile UI in the afternoon, and refactor a web component before supper. Because of this phenomenon of open sharing, it has become easier for developers to learn new things that they would normally shy away from.

Why crowdsourcing for full stack developers?

Just because developers now have superhuman capabilities to deliver anywhere up or down the stack, it doesn’t necessarily mean they want to or that they are equally efficient across the spectrum. Sometimes developers want to focus on an area of expertise or learn something new. At one time or another, all developers wish they had a lifeline for a certain topic or even just a second opinion to round out their full stack balance. That’s where crowdsourcing can help to augment a full stack developer’s tool kit, allowing them to make more informed decisions, have a better overall understanding, and increase efficiency.
When most people think of crowdsourcing software, they think of building all or part of a project using a network of people, like the Topcoder Community. While that is proven to work, there is also another lesser-known approach to crowdsourcing that is equally valuable. It involves discrete, independent challenges or tasks and a developer’s ability to quickly launch them and consume their output. This type of “wholesale crowdsourcing” addresses developers’ needs to effectively operate on parts of the stack where they need:

• A second opinion.
• A better understanding.
• A demonstration or how-to.
• A bake-off.
• … Or they simply don’t want to do it.

Often we call these types of challenges exploratory because they pose a simple question and have a variety of acceptable solutions, wherein the consensus can be more valuable than the best result. However, they can sometimes resemble a task with very specific inputs and outputs. What all of these challenges/tasks have in common is that they are isolated from any given project, are simple to articulate (sometimes in just a couple of sentences), and can be launched in mere minutes. Most of the time, the title of the challenge alone is enough to pose the central question. Here is a list of challenges that fit this description:

• Identify the best JavaScript library to generate client-side pdfs.
• Create a Swagger doc from this JSON payload.
• Provide a how-to to install Netflix Conductor on Bluemix.
• Help us select Angualr1, Angualr2, or ReactJS as a front-end framework on Salesforce.
• Create a Docker file for JBoss Fuse and two FTP servers for testing web services.
• Provide CSS based on this PSD file.
• Convert this script from Python to Node.js.
• Help refactor this code to the Airbnb style guide.

When put in good hands, crowdsourcing these à la carte utility challenges can provide an advantage to developers by complementing their existing skill sets. Ready to start a project?