Marketing for Developers

Amongst my developer colleagues, I find that I'm an anomaly when it comes my stance on marketing. For many of my peers, and I think to the general public, marketing has - rather ironically - been given a bad rep. It's seen as a finicky field: engaged in activities that are hard to measure, and at its worst, it's seen as a spurious profession. I think this reputation is the end result of a lot of bad apples that not only execute marketing badly, but as a result, have ruined it for the rest of the lot.

I very much disagree with this negative characterization of marketing. In fact, I think marketing is one of the most important functions when it comes to running a business, and especially when it comes to ventures engaged in technology. My goal in writing this is to dispel a few myths about marketing, and perhaps reveal why it is personally such a fascinating part of running businesses, especially as a developer.

Marketing is about vision

First and foremost, marketing is about vision - this is especially the case when it comes to branding. When you talk about branding, it is impossible to do so without talking about what your business stands for, its values, and what it believes in. Every great brand does this. Google wants to organize the world's information. Apple wants to build beautiful products that have technology that changes the world. Amazon wants to be the world's best marketplace and destination for commerce. Brands - and the marketing industry that shapes them - are about visions and beliefs for a company...and the people that make it up.

Most people think that branding is just about what others think about what you do. That's not the case. Companies that do branding well also understand that what you say you stand for must be authentic and reflects internally: what you say you stand for to everyone else is also heard and should be reflected by the people who you work with day to day. When the brand and the values it represents are clear, it also gives everyone a clear direction about what is important and what is not. It makes the priorities clear and makes decision making easier.

What features should we build? You can be great at making features - making them performant, solving problems your users really have, quantifying and measuring their success and optimizing them for certain behaviors. But being performant at building a product does not imply you should build anything and everything you could build well. Should Apple create amazing razors? I bet they could. I bet they could design them beautifully, revolutionize the way that we think of shaving, and destroy the existing razor market the way they have so many times with other markets. But they probably won't - it doesn't fit in with the values they have. It doesn't fit with their brand.

What design should we go with? Design is largely about communicating and building an experience. It's about what you want your users to think of you, what you want your users to feel, and how to guide your users. Should Google have designed search in a way that is monochromatic, made queries useable like SQL statements, and gotten rid of images and video in their search results? They probably could, but that's not what they stood for. Google isn't just for the technically minded - it's for everyone and their design reflects that. The important thing here is to say that "Google isn't just for programmers - it's for everyone" is precisely what it means to create a brand.

I know far too many companies that end up working aimlessly because they do not have a clear vision, let alone clear branding that reflects any such vision. They build features because it's the next natural thing to do or what will drive revenue, without thinking about how it fits into a mission and if it reflects the brand that communicates that vision. I know too many companies that design differently on different features each time because they do not have a central message they want to communicate, and I know too many companies that design too narrowly because they think they might have a brand but don't flesh it out to give it flexibility.

It's much like growing up as a person. When you're younger, you can do what you want as the circumstances direct you. You take a job straight out of school because it's the offer that is laid out before you. You date a person because they're the one available at that particular point in time. You probably do these things because you haven't fleshed out what you value yet, let alone be able to talk about it, as you would with a personal brand. If you tried to explain to someone interviewing you what you stood for, you'd realize that certain jobs are for you, and some aren't. But if you don't even know what you stand for, you can't talk about those values. If you tried to explain to someone what's important to who you want out of a life partner, you'd realize there are certain people you would and wouldn't want to date. But if you don't know what's important to you, you can't communicate it, let alone decide with those values. To be able to explain and communicate what you stand for is exactly what branding is, and it helps you make the right decisions you have when you grow up because it forces you think about what is important to your organization and the people that are working in it. The same goes for companies: part of growing up is fleshing out what you stand for, so when you explain what you stand for you have a clear answer, and when you make decisions that affect your employees, your users, and your colleagues, you also have a clear answer then.

Marketing is about revenue

Most people think that marketing is this fluffy activity that has no measurable impact on the business. Marketing seems to make and produce swag like t-shirts and toys for users to have, people say. Marketing seems to only go to events that just generates goodwill, others will say. Marketing does things that are there to "make people feel good," but has no measurable impact on the bottom line, they say. Some may even dismiss my points about vision and self-actualization as spurious. That's fine - but if numbers and hard results are what you want, marketing is also an important piece of the story when it comes to generating revenue.

In every business, you will find yourself selling something. It could be ads, content, services - any number of products. To get people to buy something from you is the ultimate goal when it comes to sustaining the business. Getting people to buy a product means money that is used to fund the rest of the business, which in turn should generate more revenue for the process of creating products and selling those products to repeat and grow. So selling your products is a very important thing to do.

However, most of us should understand that building a great product is not enough to get people to want to buy something. "Build it and they will come" is shown time and again to be a meaningless aphorism. However, I do not mean to say "tell them and they will buy" either. Building a great product is necessary but not sufficient to drive sales. What I'm saying is marketing is also necessary but not sufficient to drive sales: you need both. There have been too many times when people try to argue pointlessly that one is more important than another when you fundamentally need both to do so. Why is that the case? Because they are each different parts of a simple funnel that is core to understanding why marketing is such an important part of the organization:

  1. Someone becomes aware of you and your product
  2. Someone expresses interest in your product
  3. Someone buys your product

Having a great product is important because you need something people will buy. That's absolutely important. Sales is largely concerned with converting people from steps two to three: they are taking people that have expressed interest in your product to buy. You have salespeople calling people you have tracked that go to a product page for example, and ultimately they try to convince them to buy what you are selling. In less mature organizations, the salespeople are also trying their hardest to convert at the top of the funnel as well, from steps one and two: they'll try and prospect and get people to express interest in buying. However, that's not efficient to do.

This is where marketing comes in. Marketing is specifically interested in increasing the size of the top of the funnel that makes everyone aware of you and your product and turns them into meaningful leads that signal someone is interested in buying. From there sales can take those leads and try to convert them actual customers as they always have and focus on that function. Product is there to make sure that what the customer is buying works well and solves the problem the customer has. Superior marketing tries to optimize every part of the funnel from the top all the way down to a sale and so touches almost every part of the company at some level.

So those "fluffy" marketing activities are usually not "fluffy" at all and are in fact meant to generate meaningful leads for sales and ultimately turn into revenue. A great example that demonstrates this activity is a conference. When marketing goes to sponsor a conference, they better have a way for the people they meet to get in touch with a sales rep and maybe even incentivize them with a discount. Marketing should be able to say how many leads they generated at the conference, how well they converted into customers, how much revenue those customers were responsible for, and how that revenue was beneficial to offset the costs of sponsoring the event in the first place. That's the way you attribute meaningful results on the bottom line to marketing, and it's marketing's job to optimize and track every part of the funnel that leads to revenue they are involved in, especially the top of it. This is the same with ads marketing buys on Google and Facebook, this is the same with blog they keep bugging you about, this is why they're so obsessed with tracking and optimizing and A/B testing on so many parts of the organization: it's about providing as many entry points at the top of the funnel and tracking those visitors into leads into customers as they go through the funnel. It's about pulling all of those pieces together into a centralized system that marketing has a large role to play within.

Marketing is about user intelligence

Marketing is also the most well-equipped to understand your customers and users. In other words, marketing understands your market - it's in the freaking name for goodness' sake. When you're in startup mode, it's a very important thing to be talking to your users. You need to understand what their needs are, validate hypotheses about what features you're thinking of building, and use those insights and turn them into features that have meaningful impact.

The problem is that when organizations start to get larger, talking to users and keeping this two-way communication open becomes very difficult to do. Companies begin to isolate themselves and build features in a vacuum, especially when it has historically been commanded by product, engineering, and/or design. Your developers can't be spending time talking to users, they need to be coding. Product is concerned with making sure features are being shipped and originating ideas to solve user problems. Talking to users quickly becomes something sent to the back burner. In the worst case scenario, features get built that do not reflect the needs of the customer or simply don't work because of missing information somewhere along in the process.

This is where marketing comes in. Because the marketing team is the part of the organization that is most user and customer facing at the top of sales funnel (unless sales is right up there with them prospecting), they have the best understanding of the people using your product at a large scale unless product has a dedicated system to produce meaningful insight. This is because in order to do their job correctly, marketing should be constantly measuring, testing, and optimizing business critical funnels across the product anyways - it gives them enormous insight into what your users are thinking and how they behave. That insight should be delivered to the feature-generating side of the organization and taken seriously, or even better, they should be part of the conversation when features are being formed because they would have the best data to understand what will or will not work. As a sidebar, it's why I think data scientists are sometimes most interestingly suited to be within the marketing departments of organizations if they aren't in product or engineering - they can give unbiased, data-driven insights about users that make meaningful impact on feature decisions. It's also why I think this myth about marketing as something hard to measure is so far off base if user intelligence is done correctly.

There's so much overlap here

After reading this, I hope there's a clearer understanding of what function marketing plays in an organization: they can be essential in formulating vision, driving revenue, and understanding users. They perform these functions in a self-actualized, introspective, and data-driven way when done correctly. To minimize - and even worse, to dismiss - the role of marketing would be unwise to say the least.

For many of us that are in startups, it's very easy to overlook the need for marketing. Deeply understanding your values, turning a cool technology or community into something sellable, and becoming data-driven, are all things that many put off until you become a larger organization that is on its way to becoming mature. It becomes terribly difficult to realize that when there are such damning myths about the function of the profession and when you are so focused on building stuff.

But something I think is most fascinating about how marketing works is how eye-to-eye I think it is with how developers work. Developers love working on things that are well thought through and take a meaningful stance on issues related to technology. We stand vehemently for things like type systems and the languages we use, so why don't we boldly take a stance for the companies we work for and that we strive for like marketing helps us do with brands? Developers have the luxury of being able to work in an industry that lets them create and support their livelihood with it. Marketing plays a key role in organizations that allows us to do that by driving revenue opportunities, and also make others feel good about the things we create. Perhaps least controversially, developers love being data-driven. We love thinking about what is performing better than something else, by how much, and why. Marketers do the same, but instead of figuring out what data is moving efficiently, they think about how people are are behaving and why.

Marketing and development have so much in common when they're both done well that it sometimes mystifies me why they are sometimes at such odds and misunderstood to each other. I hope this...you know...helps positions things correctly :)

My first twelve months working at Stack Overflow

Today is my one year anniversary working at Stack Exchange. Some of you many remember a post I wrote about my first six weeks working at Stack Overflow. This is a follow up to that post, and what has changed since. It's also my first time blogging in a very long time, and as you'll read, it's because I've been quite...busy.

If you want some background, I recommend reading my previous post as well.

3 months in - a big project and many hats

The three month mark at any job seems to be one of those turning moments. Before the three month mark, it's a lot of trying to get a foothold in the culture, learning to be comfortable in a new codebase, and so on - as I made apparent in my previous post. At three months, the newbie sheen starts to wear off, and you actually have to roll up your sleeves and get some serious stuff done. This is especially the case at a place like Stack, where you're working with - quite literally - some of the best technologists in the world, and there's quite a bit of pressure to ship something interesting you can take ownership of.

For the city page feature I was working on, this was the picture I used for the hero image for Washington D.C. Of course, this is not the Capitol building, but St. Paul's Cathedral...in London. This was quickly resolved.

For the city page feature I was working on, this was the picture I used for the hero image for Washington D.C. Of course, this is not the Capitol building, but St. Paul's Cathedral...in London. This was quickly resolved.

My first big project was working on a feature to let developers discover what technology is like in a particular city. The idea being that developers have a lot of freedom to choose where they would like to work, and it is usually largely determined by where a company and the developer's life will be located geographically. It was something I had expressed interest in working on quite early, but this was actually a project that was being worked on before I had joined. I was later pulled in to help another developer with the project, since it dealt with particularly nasty issues around...geocoding.

Now if any of you have worked on geocoding intensive project, you will know that suddenly being an expert in geocoding is both a blessing and curse. It became one of my domains of expertise very quickly (having some prior experience helped), as the developer that I had been working with moved on to help with a mission critical project. I was suddenly working on a large feature without another, more senior developer working with me. But good things came out of necessity: it really started drawing out what I thought were areas of expertise that had been dormant before.

Taking "full stack" to a different level

A quick detour. Stack hires only full stack web developers for the most part. For those of you that don't know what being full stack means, it means that a developer is at least proficient in all the essential parts of web development: front end, back end, databases, some systems administration, security, etc. Now, some of these domains of expertise are somewhat uncontroversial: being full stack probably implies you know at least one back end related language like Python, C#, or Java, for example. Other areas are more arguable: design, sales, management, etc. The degree to which one is full stack has a pretty clear floor, but there's no definite ceiling to what areas are afforded being "full stack" that is still clearly in the domain of being a "developer."

This is something I became very aware of after the three month mark. I know I am not the strongest developer in a strictly technical sense. As I had mentioned in my previous post, I had switched technologies entirely: many parts of development, sometimes what seem like rather silly and trivial things, still escape me. That being said, I believe being able to put on a lot of different hats in the company as a developer has helped me tremendously.

Being able to take the technical expertise and talk about product like a product manager with the product manager was extremely helpful to see eye to eye on major feature decisions. Interesting ideas that had crossed my mind instead came out of the product manager's mouth, and it became a really delightful experience to work on features I wholeheartedly agreed with and felt a part of.

Having experience with design helped in numerous ways: getting further along in prototypes so there's less heavy lifting done by the designers, talking about systematically doing front end work throughout teams at a higher level, elevating design conversations beyond aesthetics and more about vision and features and experiences. The list could easily go on - the designers are a great pleasure to work with.

I also feel I have a rather unique relationship with the marketing and sales teams. A large part of this is because I do a lot of evangelism and tech education work (more on this later), but most importantly, it's because I think the vision and narrative of Stack Overflow as a brand and sellable product comes directly from my colleagues in these areas. For me, development is largely in service to an ultimate vision supported by a strong business model. Because of that, I feel I spend a unordinary amount of time (at least to my technical colleagues) working with marketing and sales.

After the three month mark, I was developing areas I was particularly well suited for within the organization, and having a large project helped tease them out. It was also becoming more apparent to myself, and I think to others, that I wasn't a strictly technical developer: having had some entrepreneurial experience let me feel I could offer more than just quality code.

6 months in - pushing a bit

At six months, I had finally pushed my first major project. It was a feature I was proud of and could really take ownership of, but it also marked a turning point for myself. I felt like I had overcome much of the imposter syndrome that sometimes comes with working at a place like Stack, especially given my background. I'd only been two years out of college, without a computer science degree (a philosophy degree instead), without even a technical internship under my belt, and as a self-taught developer to boot. Being able to push a feature on something as big as Stack Overflow was huge for me.

It also did something else: I felt I could actually start pushing the organization a little bit. As great as an engineering organization Stack is, there were things I thought I could help improve. After my first initial project, especially since I had been working on it for so long, I started working on smaller projects while tightening up the large geocoding refactoring effort that I had started.

Many of these projects were related to what I felt were sometimes overlooked areas: front end organization, automated environment set up, and allowing marketing to get access to more content control. It also helped that my mentor who had helped onboard me was working on these same projects - it still strikes me how eye to eye we see many things. I was starting to let some of what I really wanted Stack to become begin manifesting itself as projects that I became a part of or started. 

(D)evangelism

At about six months in was also when I really started ramping up my outside involvements with Stack. This was for a number of causes. Since I joined the company, diversity in tech has become very pressing and widespread issue (thankfully). I think that Stack is in a particularly interesting situation both externally and internally when it comes to diversity issues. Externally, we have helped forge the largest community of developers in the world and have a real opportunity to help steer the course of diversity at large. And internally, for what it's worth, we are at least particularly well represented on the LGBT front (if you know anything about me, or if not, at least about our CEO, Joel). We're also very committed to supporting more women in technology, not only within the company, but for the broader community as well. I am also particularly, and quite naturally because of my work with Bento, committed to the broader mission of teaching people to code. These are just a few examples of what I and others in the company are working on with this effort.

Hand in hand with this push was a broader effort on my part to evangelize Stack Overflow as a developer. Now, you might be thinking, why in the world would Stack Overflow need developer evangelism for if so many developers already know and actually use the site on a daily basis?

This was the line for students to get Stack Overflow t-shirts at MHacks, the first hackathon I pushed for us to sponsor. It was 200 people long!

This was the line for students to get Stack Overflow t-shirts at MHacks, the first hackathon I pushed for us to sponsor. It was 200 people long!

The short answer is because I truly, deeply believe that Stack Overflow is in a unique position to lead and better the developer community as a whole. Teaching people to be better developers with Q&A, inspiring minorities in tech to code and get involved, improving the state of tech recruiting...the list goes on. To get people to rally around the mission of Stack Overflow in addition to the technology is why I think evangelism is so important to me as part of being a developer at Stack. Who better to rally a community of developers than other developers? I wanted to be a part of that, or at least take a first step forward on it.

I started doing "devangelism" pretty quickly, especially as I became very close with marketing. I was going to student hackathons, speaking on panels, hosting events at the office, taking weekly lunch meetings with students...the list goes on. I think at one point I was averaging two or three talks a week on top of my developer responsibilities, and I've been dubbed "St. Jon the Devangelist" at one point. It's really amusing, and one of my favorite parts of being a developer at Stack. I hope that one day Stack formalizes a real developer evangelism program, but up until that happens, I'll probably continue doing what I can to further the mission.

9 months in - full swing

At the nine month mark, I felt like I was actually starting to go full speed with my work. The last three months have really been about taking the lessons I learned in the first nine and running with them. I feel like I am familiar with my strengths, and can really focus and hone them. I found myself wearing a lot of hats, at once, and pushed very hard to get multiple projects through more aggressively. My wardrobe now largely consists of Stack Overflow swag, and I'm somewhat guiltily happy about it. I've recently started helping with a large push to bring up the level of design with Stack Overflow Careers, something that I'm very happy to finally see happening after so much time working with design. I'm leading projects around user acquisition and marketing that I'm excited about and have wanted to do since I started. All in all, I'm doing work that I enjoy and feel like I'm making an impact on the organization and the amazing community it supports, whether it's recognized or not.

That actually brings me to my last point.

The impact

If there's one, overarching, incredible thing that I really love about working at Stack for the last year is how I know I'm working on something that helps millions of people around the world. We are not a very large company relative to the impact we make. I think because of that impact, we attract some of the best people to the company, and it's such a privilege to work with them every day. It's also because of that impact that I get to work on unique challenges that I wouldn't get to anywhere else so early in my career.

But throughout my first twelve months here, the moments I enjoy most are when I talk to strangers about my job. It's a hit or miss, saying I work for Stack, because you can be non-technical and have no idea what I'm talking about, or you're a developer and it's a completely different experience. I don't think I've ever been hugged for telling someone where I work. I certainly don't think I'll have that happen to me multiple times with another company like it does now.

TL;DR I can't think of a better place for a developer to work at, certainly not for myself. The first twelve months have been life-changing. I expect the many coming months to be more and better of the same.

Finally, a guide to hosting your website

So you now know HTML, CSS, and JavaScript. You've done the interactive tutorials, watched YouTube videos, and even picked up a few books. You know why inline-styling is bad and discovered the wonderful library that is jQuery. Then it dawns on you: you've learned the languages you should know, but you still have no idea how to take that code and turn it into a site on the Internet for the world to see.

Since I built Bento, I've heard this complaint time and time again, and it's a shortcoming for most learning resources on the web (even the ones that I admire and love). So here's a (lengthy) guide about how to write static web pages on your computer then put it on the Internet.

One important note before we begin: this covers a very large number of topics that will get you to hosting a very basic website. This guide does not cover how to host a website with a back-end web framework, databases, caching, and so forth. I assume that since you are a beginner who needs this guide, your skill level is far more fundamental and you know only HTML, CSS, and maybe some JavaScript. I will probably write another post on hosting more sophisticated sites later on.

Let's get started.

How to build and test a simple site on your computer

The first thing you need to know is how to get your HTML, CSS, and JavaScript running on your computer without having to do it inside of an interactive tutorial like in Codecademy or Dash. It's fairly simple:

  • Open up a text editor. Notepad or TextEdit works just fine, but I highly recommend you download a specialized editor like Sublime Text to make your coding life infinitely easier.
  • Create a new file and type in the following HTML:
<html>
<head>
</head>
<body>
<p>Hello World!</p>
</body>
</html>
  • Save that file as index.html on your Desktop. It's very important that the name of the file after period is "html" - this part is known as the file extension. You might have noticed that images have different file extensions like jpg, gif, or png. This is just like that and it specifies what kind of file you are creating.
  • Open up a browser like Chrome, Safari, Firefox, or Internet Exploder. Go to File > Open File and find the index.html file you just created back on your Desktop.
  • You should see "Hello World!" in your web browser. Try changing the contents of your index.html file with new HTML, save the file, and hit refresh on your browser. You'll see that it starts changing! 

What just happened there? Your browser is actually taking the text in that HTML file and interpreting it as HTML code, turning those tags into actual images, applying boldness to tags, and so forth. The browser is just taking text from a file and interpreting it as instructions.

What about CSS and JavaScript?

CSS and JavaScript work in a very similar way. You create files like index.css (for files containing only CSS) and index.js (for files only containing JavaScript) with new file extensions to specify what kind code is in the file. So try doing the following:

  • Create a new file in your text editor and type in the following CSS code:
html {
color: blue;
}
  • Save the file as index.css on your Desktop (notice the new file extension, css). It needs to be in the same directory as your index.html file (for now).
  • Create another new file in your text editor and type in the following JavaScript code:
alert("This is an alert I created in index.js!");
  • Save the file as index.js on your Desktop (notice the new file extension, js). This file also needs to be in the same directory as your index.html file and index.css file (in this case, the Desktop).
  • Finally, change your index.html file with the following code and save it.
<html>
<head>
<link rel="stylesheet" type="text/css" href="index.css">
</head>
<body>
<p>Hello world! This text should be blue because of the CSS I wrote in index.css!</p>
<script type="text/javascript" src="index.js"></script>
</body>
</html>
  • Open up index.html in your browser just like you did before (or refresh it if you already had it open). You'll see that your JavaScript gets executed, your CSS rules have been applied, and your HTML is there too!

HTML files work in concert with CSS and JavaScript files. The <link> tag that you put in between the <head> tags allows you to refer to CSS rules in another file. In this case, it is index.css. You could have renamed your CSS file to anything you want as long as it had the CSS file extension after it. It could be dog.css or even i-dont-care-what-I-call-it.css and you would just need to change the "href" attribute in the <link> tag to make it work. It is generally good practice to put your CSS files within the <head> tags so that your style rules are loaded before anything after it shows up in the page. Otherwise, your content may look like ugly default HTML without styles for a split-second while the CSS is being loaded.

This ability to reference code in other files works exactly the same way for JavaScript except with the relevant <script> tag is right before the ending <body> tag and uses the the "src" attribute. It is generally good practice to put your JavaScript files at the end of your <body> tags so that it won't block the loading of HTML or CSS. Otherwise, it may give the user the impression that your site is very slow if you have a lot of JavaScript.

File Structure

At this point, it should be clear that all you need to do when building a site on your computer is writing HTML, CSS, and JavaScript into text files with the right file extensions and firing them up in a browser (very simply put). However, putting all of your files onto the Desktop would become very cumbersome very quickly, so you should put your files into separate folders and even different types of files into different folders. Something like this:

hello.jpg

Typically, what ends up happening is the following:

  • All of your site's files are put into a single folder. This includes HTML, CSS, JavaScript, and even images.
  • Your HTML files typically stay at the root of the folder while your other files go into folders separated by type. I typically create folders named cssjs, and img for CSS, JavaScript, and images respectively.
  • Your HTML needs to be modified to include the right paths for your different files if they are in different folders. For example, if you used the same folder names I did, your HTML would look like this (note the href attribute in the tag and the src attribute in the tag):
<html>
<head>
<link rel="stylesheet" type="text/css" href="css/index.css">
</head>
<body>
<p>Hello world! This text should be blue because of the CSS I wrote in css/index.css!</p>
<script type="text/javascript" src="js/index.js"></script>
</body>
</html>
  • If for some reason you needed to reference a file that was one level above the current folder that file is in, you would use two periods ".." to indicate the folder above:
# If index.html were in a folder and index.css were in the folder one level up:
../index.css

# If it were in a folder two levels up:
../../index.css

# If it were in a folder named 'css' that was in a folder one level up:
../css/index.css

Why "index" for the file name?

One last thing: you might be wondering why the HTML file is called index.html as opposed to anything else like home.html or puppysareawesome.html. This is because the default file to go to when accessing a folder in a browser - particularly when it hosted on a server - is index.html. This will be more apparent later when we actually publish your code.

How the Internet works, simplified

Now that you know how build a simple site on your computer, it's time to take that knowledge and apply it to publishing it on the web. Pay attention here, because what you're about to read is probably the most important part of this entire guide. You need to realize is that the Internet is just a bunch of interconnected computers, and when you visit a site, you are using code that is on someone else's computer instead of yours.

Let that bit of knowledge sink in a little.

This is a vast oversimplification, but you've seen how code is practically interpreted text in files with special file extensions. You could have accessed those files from someone else's computers rather than yours. You just don't know how to put your code in a place that's available to everyone yet.

When I type in "http://www.google.com/" in the address bar of a browser, a number of things are happening very, very fast. In very simple terms:

  • The first thing that happens is that a request for files is sent to a particular computer. That web address you type in "http://www.google.com/" is just a nickname for something known as an IP address that looks something like 127.0.0.1 . That IP address is like a mailing address that points to a particular computer where the files are coming from.
  • When the computer is located, the right files that contain the HTML, CSS, and JavaScript are sent back to you and the browser.
  • The code that is in the HTML, CSS, and JavaScript files is then interpreted by your browser (Chrome, Firefox, Internet Exploder) and used to display what you see in the window, just like you saw with the local files you created earlier.

What is effectively happening when you access a website through a browser is that you're accessing code on someone else's computer that was sent to you instead of from somewhere on your machine like your Desktop. The main difference is that these are not the ordinary kind of computer like the one you're probably using to read this guide. These computers are known as servers and are designed to do exactly what their name implies: serve web pages. What that means for you as someone who wants to build a website is that you need to put your code somewhere where other people can access it too - on a server. There is a great video to watch about how this process works that I feature in the fundamentals portion of Bento - it's the first link on the whole site because it's so crucial to understand how this process works:

What you need to learn now

At this point, there are two major parts to getting your code running in a way that you're used to with other websites. The first part is actually putting your code to a server. The second part is getting a domain name that points to the server that you're putting your code on.

Putting your site on a web server

There is an endless number of combinations of methods and services you could try to get your code hosted on a web server. However, it's simply not useful to enumerate most or even many of these combinations at the skill level you are at if you're reading this. Therefore, I will provide only three methods and strongly suggest one.

Traditional web hosting (don't do it)

There are a lot of services out there that will essentially let you rent space on their servers and provide you with some useful tools on top of them. Traditional hosting is usually provided by many popular domain name registrars like GoDaddy and simply provide you with a certain amount of space that you can serve files from. For the majority of these services, you simply upload your files and folders to the server just like you would upload an image from your computer to Facebook.

Many of these traditional hosting solutions will also let you upload files via FTP (File Transfer Protocol). This lets you upload files directly from your computer and download files from the server without having to go through the hosting service's website.

My advice about this sort of hosting is: don't do it. This method of uploading files manually is outdated by at least a decade and has a number of issues plaguing it: lack of flexibility, difficulty of use and configuration down the line, and generally very expensive. The only reason I mention it here is because you will inevitably have it shoved in your face when you buy a domain or search the Internet for hosting services. Just walk right past it.

Cloud services (overkill)

Another thing you'll probably run into are cloud services with names like Amazon Web Services or Heroku popping up. These are ways of hosting your files in a way that lets you fine tune things to scale a website, determine where the servers you want are geographically, and a whole slew of other options. My advice for you here is to avoid these solutions if you are building a site that is just HTML, CSS, and JavaScript. Such sites are also known as static sites if you weren't familiar with the term. Assuming you don't know a back-end language like Python or Ruby and their accompanying web frameworks, these cloud services are overkill. Since this guide is meant for people with little to no experience publishing their code, the skill needed to manage these services on top of writing good code is beyond the topic at hand.

In the future, I plan on writing another post to talk about how this could be done, but it's an entire other section of its own. So assuming that you are building a static site, best to avoid these for now.

Instead, use GitHub Pages

Okay, what you should do is use GitHub Pages. Full disclaimer: I do not work for GitHub and GitHub is not paying me. I personally think it is the best way for a beginner to publish their code for a number of reasons. It was even used to host the original version of Bento before I switched over to cloud services. Here's a few reasons why I think GitHub Pages is a good idea:

  • It requires that you learn Git, a popular version control system (though not the only one out there) that is an essential tool for any developer that hopes to work in teams or for a company. Git is one of the most popular technologies for version control out there.
  • By learning Git, you will also have to learn how to interact with the shell. This is another basic technology that lets you interact with your computer in a very fundamental way.
  • Hosting this way gets you started building a GitHub profile, which allows you to open source your code so that it can be seen by others - especially potential employers and other developers. I cannot stress enough how important this can be if you want to advance your career, especially as a beginning developer.
  • It's absolutely free to host your code on GitHub and have it be published in an accessible way.
  • It is shockingly easy to get your code published.

Now about that last point about it being shockingly easy: what that means is that you might have to pick up a few new (but easy) skills along the way if you're a complete beginner. This is what I recommend:

  1. Learn how to interact with the shell on your computer. Here's the Bento page for it. I highly recommend you start with the first link Learning the Shell.
  2. You need to learn Git. Here's the Bento page for that. I highly recommend you start with GitHub's own tutorial on Git, Try Git, which is the first beginner link on the page.
  3. Once you learn how to use the shell, install Git on your machine, and how to use Git, you need a GitHub profile. Go to http://github.com/ and create a profile.
  4. Create a new repository for your code. This should be in the folder that contains all your HTML, CSS, JavaScript that you want as part of your page. In other words, you'll have to "cd" into your folder from the shell, and run "git init" there. I'm assuming you're following the directions in the link from GitHub for the next steps, so make sure you get to the stage where you're going all the way to pushing your code to the newly created repository.
  5. Publish your code by pushing your code to a branch called gh-pages in your repository. I'm also assuming here you know what git branching is - if not, click here for a tutorial used on Bento. The full instructions on how to do this from GitHub are very good, so make sure the read them.
  6. Once you've successfully pushed code to the gh-pages branch, you should be able to visit your site at: http://<github username>.github.io/<repository name>. For example, my GitHub username is jonhmchan and I could host my code in the gh-pages branch of a repository named bento. That means any code that commit and push to the gh-pages branch would be seen at http://jonhmchan.github.io/bento

That's really it! Assuming you know how to use the shell, Git, and GitHub, all you need to do push your code to a branch called gh-pages and GitHub takes care of the rest! At this point, there's probably one more question lingering in your mind: how do I get a fancy domain name like www.bentobox.io if I don't want to use GitHub's auto-generated web addresses? That's where buying domains come in.

Getting a domain name

The next thing you'll want to do is get a domain name. A domain name is how people will find your site: Google's domain name is google.com and Bento's domain name is bentobox.io. You'll have to register your domain name with any number of registrars like NameCheap, Gandi, or GoDaddy. Usually, these domains will be very cheap, sometimes as low as $10 a year. However, be careful with how you search or buy your domains though. Here are some things to keep in mind:

  • Getting the perfect name is going to hard. A lot of the great names have already been taken, especially on very common top level domains (TLDs) like .com and .io. Chances are, you're going to have to be pretty creative with your domain name unless you have a lot of money to shell out.
  • The best way to check if a domain name is taken is to simply try and visit it. Some domain name sellers will actually try and buy up domains that are typed into their sites just so they can be sold at a higher price because you've indicated demand. You can also try doing what's known as a "WHOIS lookup" to find out if the domain you want is registered (just Google it).
  • When you buy a domain name, it is required that you provide information about yourself to the WHOIS database. This is a public database that is searchable. You can get around this requirement by using Domain Privacy that provides stand-in information for the WHOIS database and forwards information along to you. Some registrars like Gandi automatically do this for you (I use Gandi for pretty much all of my domain purchases), but services like GoDaddy only make it available if you pay for it on top of your domain. Be sure you know before buying your domain.
  • Buying a domain that's already taken is very hard to do. Many of these domains are taken by squatters who buy up a lot of domains just in case someone wants it and will sell it at a higher price. Negotiating to buy that domain also takes up a lot of time and effort, and probably a lot of money if the domain you want is valuable. In most cases, you're better off just choosing another domain name.

Once you actually get a domain that you're happy with, you'll have to change some settings to point to the servers your files are hosted on and it's a bit different depending on which method you're using to host your files. That being said, the vast majority of cases require that you simply change a few options on what's known as DNS records - which you can get a brief overview of here. However, if you are using GitHub Pages as recommended earlier, you can read GitHub's own instructions on how to set up a custom domain for your pages here.

Also, you should know that changes you make to your domain settings may take a while to take effect. If you're running into problems, just check back in an hour or so. If it takes more than that, something might be up.

That should be it

I hope this serves as a starting point for someone just learning to code to actually publish your work. I cannot stress enough how important it is to get your code out there in the world. For so many developers I've had the pleasure of meeting, being able to create something that lots of people can use is their favorite part of learning to code. Even for me, that feeling of creation is what got me addicted to development. 

Now go build stuff and share it with the world.

Designing a @#$%ing address form

Follow me on Twitter

Address forms, at surface, seem like a common enough thing on the web that it should be relatively simple to implement. Anywhere where something needs to be shipped, paid for, or organized in the real world, the form almost always appears. Yet, when the moment comes that you need to implement one yourself, it can quickly become a journey into a wilderness plagued with feature creep, standardization pitfalls, and in some occasions, death (not really, but I have ended up with some massive headaches at the end of the day).

At Stack Overflow, I've been tasked with making an address form that would dwarf many other address forms in functionality. If it were simply a few text fields to be stored in a database, that would be no problem - but the form we wanted did all of the following:

  • Internationally useable and localized
  • Suggested, validated, and autocompleted in real-time
  • Geocoded the address down to the highest resolution possible (street is best, city is required)
  • Allowed the user to correct any of this information, including a map that changed the geocoded information

This is my journey in building such a form. Hopefully, it will give other developers a bit of guidance when they almost inevitably come upon this sort of thing in their projects. I've had to do this on a major project at least a dozen times now, and it seems to be a feat each time.

Needs moar fieldz (not really)

First, the absolute basics. Most of us have a pretty good understanding of what an address is, and how it should translate into a form. Consider the following:

standard_form_1

Rather innocuous. Most people in the United States would consider this a pretty standard address form. It has all the components that one would need to send someone something over the mail, but after even thinking about it a little, one could see that there might be some fields missing: a second street address for apartment numbers, floors, and suites for example.

2.png

A small change, surely. But wait! Some addresses require that you also need a name for a company like when you need a c/o, and we wouldn't want people putting that in the first street address line then their actual address in the second line, right? We should tweak our design to make this clearer:

3.png

That's better. But wait, we could make this address form even better by abstracting away some of these elements into smaller components, right? Name should be first name, last name, street addresses can be broken up into street lines and floors - those should all be different fields! And what about those nasty 9-digit zip codes? That should be broken out too, but we'll make some of it optional. Oh, and while we're at it, we should be able to make the State field a drop down with abbreviations of states in the US, because who wants to waste precious moments of their lives typing in two letters?!

4.png

Perfect! This is exactly what we want for address form. You start patting yourself on the back at how standardized and abstracted this form is, and you think that there couldn't possibly be anything wrong with it. Until, of course, your boss comes around and goes: "Holy s@#t that's a scary form. No customer is going to want buy something from us if they have to fill that out. Oh, and I also came by to tell you that we're now taking international orders, so I don't think that 'state' field will work anymore."

You stare blankly at your screen with the same look a five year old gets when he drops his ice cream right after leaving the shop. Some places don't even have the concept of a state, you think. Well, time to start over.

The Lesson: Don't Get Crazy

Don't try and abstract and generalize at the cost of overcomplicating your address form. Try to keep the design of your form as intuitive and as familiar as possible. Overly abstracting and generalizing is a habit that many programmers (and sometimes especially the best ones) turn into a crippling pitfall. It not only makes your form looks scary, but it probably means that it's going to be wrong because you might need to internationalize it anyway. Addresses are nasty bits of data. As a testament to how not standardized addresses are (especially once you look around the world), I defer to Falsehoods Programmers Believe About Addresses by Michael Tandy. I'll take a few of my favorite excerpts out as examples:

  • When there is a building number, it will be all-numeric.

    Counterexample: 1A Egmont Road, Middlesbrough, TS4 2HT

    4-5 Bonhill Street, London, EC2A 4BX

  • No buildings are numbered zero

    Counterexample: 0 Egmont Road, Middlesbrough, TS4 2HT

  • Well, at the very least no buildings have negative numbers

    Guy Chisholm provided this counterexample: Minusone Priory Road, Newbury, RG14 7QS

    (none of the databases I’ve checked render this as -1)

  • Addresses will have a reasonable number of characters - less than 100, say.

    Not when organisation and department names can be included in addresses! For example: Department For Environment Food & Rural Affairs (D E F R A), State Veterinary Service, Animal Health Office, Hadrian House, Wavell Drive, Rosehill Industrial Estate, Carlisle, CA1 2TB, United Kingdom

    Another example: The Gynaecology Cancer Research Unit, Department of Obstetrics & Gynaecology, St. Bartholomews & The Royal School of Medicine & Dentistry, Charterhouse Square, London, EC1M 6GR, United Kingdom

  • But street names will be reasonably short - certainly less than 50 characters

    Susanne Schmidt provides the longest street name in Germany: Bischöflich-Geistlicher-Rat-Josef-Zinnbauer-Straße in 84130 Dingolfing, Bavaria

    Graham Rhind suggests this 89-character street name in Bihac, Bosnia: Aleja Alije Izetbegovića Prvig Predsjednika Predsjedništva Republika Bosna i Hercegovina

Get it? Okay. Now, in some cases, one may even be tempted to use a single, general address line that is so streamlined that it would make Jony Ive moan in wet, glorious awe:

5.png

I am personally a big fan of this approach since I can still get a lot of interesting information about the address using some parsers (for the United States anyways), but this may be too minimalistic for most sites, even for Stack (it's really important we got cities for what we wanted). So we settled on a simple address field with another field for city, state, county, zip bundled together as "City," as here:

6.png

Are you sure that's an address? Where is it?

When it comes to validating, autocompleting, and geocoding addresses, this became a challenge in and of itself because of the restrictions on what services we could use. This was especially difficult because we decided to use MapBox (which is freaking beautiful) for our maps, which meant that we couldn't use any of Google's Maps API. Apparently, if you use any of the Maps API data, you must display a Google Map. This bears repeating in a pretty obvious, unambiguous way:

You must not use or display the Content without a corresponding Google map, unless you are explicitly permitted to do so in the Maps APIs Documentation, or through written permission from Google. In any event, you must not use or display the Content on or in conjunction with a non-Google map. For example, you must not use geocodes obtained through the Service in conjunction with a non-Google map. As another example, you must not display Street View imagery alongside a non-Google map, but you may display Street View imagery without a corresponding Google map because the Maps APIs Documentation explicitly permits you to do so.
— Google Maps API Terms of Service 10.1.1.g

 

Considering how comprehensive Google's API is for doing this sort of thing, this was more than a trivial obstacle. I still think that the end result was still worth tying our hands together for it, but it was a rather involved process.

That being said, most geographic APIs worked pretty similarly and could do all of the data intensive things we wanted. All we would need to do is send an API a string query that represented the full address, and each service would usually return an array of formatted objects that returned its best results. Each of these objects typically involved the following points of information:

  • Formatted street address
  • City name
  • State / province name or appropriate abbreviation
  • Country or country code
  • Postal code
  • Latitude and Longitude

Now the problem, of course, were the nuances of each API. Here's a list of APIs that we considered, and those that are in bold are the ones we ultimately ended up using in our implementation:

There's also another comprehensive list of geo services on the Texas A&M site that you should check out if you're interested. Obviously, I won't be going into every one of these services (I didn't personally test them all, others on my team did). In the end, we used multiple geo services to make sure that if one failed, we could fall back on another. After testing each of these services with actual user input, we arrived at the ones bolded above. There's a few reasons we came to what we did:

  • We wanted street resolution for geocoding, and if that failed, an approximation - Not all the services give an exact latitude & longitude for an address. Some of these services (notably Yahoo) only give the center of a bounding box that represents the general area that the address may be (according to the city/state/zip most likely). So to accomplish what we wanted, we resorted to (1) SmartyStreets, (2) Open Street Maps, and (3) Yahoo Maps in that order. We considered any addresses that failed in all three services to be invalid addresses.
  • These aren't necessarily the "best" three services - This was specific to our implementation (e.g. we were already deeply using one of the services), and what worked well in this combination and order to get autocomplete, validation, and geocoding the way we wanted.
  • SmartyStreets is amazing (for US addresses) - Since a large number of addresses we were dealing with were US addresses, SmartyStreets' API did most of the heavy lifting for us. They have an incredibly straightforward service that gives extremely accurate results for the United States. Also, we opted to only autosuggest for US addresses using SmartyStreets (they have an amazing jQuery plugin for it) since it was more of a decorative helper rather than a mission-critical feature. We did automatic validation using the other services internationally.
  • Open Street Maps is pretty good for international street geocoding - Because international address formats are so varied (as mentioned earlier), it seems logical that crowd-sourced geo data would be sufficient for our needs. Open Street Maps is also used by Foursquare (which I'm a big fan of, and with whom I consulted) and it's nice to be able to get some community driven data in there.
  • Yahoo is strong at coarse geocoding - It was an absolute requirement for us to get at least city level resolution for our addresses. Yahoo isn't capable of street level geocoding, but it was sufficient for what we wanted in our previous version of address forms. It was also deeply integrated with our existing codebase, so it didn't make sense to rip it out for something different.

In the end, we bundled all of these APIs into an internal service that called them in succession until we got a list of successful results. Each of the results included at least an approximate latitude & longitude that we could use as a starting point on the map input we had. The results started suggesting as part of the first "Address" field but tried to suggest a full address rather than just that field. If they picked a suggested address, it would automatically populate the "City" field and put in the correct zoom, center, and pin on the map input. The user could, of course, modify any of these fields before submitting the form. They could even move the pin around on the map to get really precise (something we were inspired by from Foursquare). We thought that even the best geo coders out there couldn't compete with user-verified input. Here's what the final result looks like:

7.png

Final Thoughts

My team is still working on this, and it could certainly improve given all the different services available. I've yet to see how this will stand up once we actually get user input at scale. I do think we have something that works reasonably well though. It fits all the original requirements at hand, and comes to a sufficient level of standardization and accuracy that is useable for us. I would love to hear suggestions for other geo services and what other problems people might have run into designing their own forms.

Hopefully, this insight will make the perilous journey of addresses a little less troublesome for you.

My first six weeks working at Stack Overflow

Follow me on Twitter

I started working at Stack Exchange (which many know for/as Stack Overflow) as a software developer just six weeks ago. This (lengthy) post is about a number of things: what it was like relearning a lot of what I know about web development, the challenges and resources I encountered doing so, and a few pointers others might benefit from on-boarding on a new development team.

About Me

I think to grasp what this experience was like, it's useful to have some context. I won't go into all the details about joining (that's a whole other story), but just the necessary bits. Before I joined Stack, I was working at a Big Four consulting firm in their technology division straight out of college. After I built Bento, I had the opportunity to join Stack, and I took it. What's also important is that I'm a self-taught developer. Pretty much everything i know is through online tutorials and some books.

Week 1 & 2: Learning Stack's stack

Here's a quick breakdown of the technologies I was used to before I joined Stack, and what I had to pick up:

What I was used to

  • Python, Tornado
  • Sublime Text, OS X
  • Mongo, Memcached
  • HTML, CSS, JavaScript
  • jQuery, Angular
  • Git, GitHub

What I was learning

  • C#, .NET
  • Visual Studio, Windows 8
  • SQL Server, Linq, Redis, Elasticsearch
  • HTML, CSS, JavaScript, Razor
  • jQuery, Angular
  • Git, Kiln

The major piece, of course, is in the backend: I was basically changing stacks entirely. I was going from a dynamically to statically typed language, from a document-based to traditional relational database, among other things. In short, there was a lot to learn. Of course, I had the help of the whole development team I was part of, and they were immensely helpful through the process (they still are, thanks guys).

Fortunately, I spend a lot of my free time thinking about how to learn new technologies. I know I learn best through video tutorials, and I went to my usual source for learning established platforms: Lynda. They have excellent courses on C# and .NET, and I was spending most of my first week going through those videos and following along. This wasn't nearly as difficult as I thought it was going to be. I'm of the opinion that most programming languages are conceptually pretty similar, just with different syntaxes and tools: arrays are now lists, for loops are largely still the same, there are a few more features for OOP, etc. The same goes for web frameworks: there are still the basic concepts around routing, templates, and deployment. An excellent testament to this thinking is Learn X in Y Minutes, a resource I feature extensively on Bento.

Don't be mad about type systems - have a bacon donut

Don't be mad about type systems - have a bacon donut

Another major hurdle was changing development environments. For the first few weeks I was using a co-worker's previous machine, and half the battle was just getting everything set up and learning my way around the tools available.  Stack is pretty heavily Microsoft-based, and even switching over to a machine running Windows 8 caught me off guard at first. I missed the terminal window and Sublime Text and all the other tools I was used to. But after those two weeks, I was pleasantly surprised by how quickly I could adapt. There are still some things in Visual Studio that I wish I had from Sublime (I really miss Cmd+D), but I'm sure there's still lots to learn about VS that will make me just as or even more productive than before.

One of the largest hurdles was switching database technologies. Of course, I knew what relational databases were and how they worked having used LAMP for a long time, but getting back into the swing of things after a few years proved difficult. SQLZoo was a great resource for relearning some database basics, and picking things up was kind of like getting back on a bike you left in your garage for a while.

Overall, in terms of learning, it was very similar to picking up any other stack I've wanted to. I think the biggest takeaway was just taking a peek at a faction of development I didn't really touch, and for some others, even shunned. I was switching sides on some age-old wars: Mac vs PC, dynamically vs statically typed languages, unstructured vs relational databases - and I have to say I've been extremely impressed by how efficient everything is. As much as it may have pained me just a few months before to say this: I think Microsoft-based stacks get a pretty bad rep that's not entirely deserved.

Weeks 3 & 4: Learning Stack's version of the stack

At this point, I felt pretty comfortable in the technologies in their "platonic" form, but I still didn't understand how everything was organized in Stack's actual codebase and configuration. Learning C# and .NET was just the beginning. Figuring out how it was done to make Stack Overflow what it is...that's s a completely different adventure. There are a number of different teams that work on different parts of the site, but I work specifically on the Careers team. It's newer than the core of the technology running the Q&A sites, and luckily there are more developers physically on site that I could poke when I had questions. Since I was working on an exercise project, it gave me plenty of time to explore and experiment without breaking anything. I also made the critical decision to branch the whole codebase and work inside of the existing environment for my project. Learning C# and .NET in an isolated way was not going to help me - what I needed to know was not about the technologies per se: it was about Stack itself.

One of our Town Halls - look at the baby!

One of our Town Halls - look at the baby!

Basically, the questions I had at weeks 3 & 4 were about how we did things: What data are we indexing in elasticsearch? What does this column in this table mean? How are things distributed on the CDN? etc. A lot of this was how Stack was doing their version of the technologies we were on - things I could only learn by asking a lot of questions and wandering around the codebase for.

Another thing was picking up style. One of the things you can't get just by borrowing from another language was best practices around a particular platform. Sure, I still avoid repeating myself and try and write efficient code, but I needed to figure out how to follow those principles in a new environment. When should I use an array versus a list (should I ever)? What's the best way to serialize an object into JSON in C#? Those were things that, of course, Stack Overflow itself was helpful for.

The most important thing after scouring the internet and bothering my coworkers: mimicking. There was already a lot of code that I could study, and simply by following the conventions that were already followed in the existing codebase was incredibly useful. When you start noticing that "var" is used more often than you'd think, you start using it. When you notice that you need X, Y, and Z classes and interfaces to render a particular route, you start doing that with your own routes. By emulating and experimenting, I picked up most of the knowledge I needed.

Weeks 5 & 6: Jumping out the nest

It helped that my Mac finally came in after a 4 week delay. The beauty.

It was now time to jump off the deep-end. Stack has come up with something pretty ingenious to handle immediate issues. Every week, there's a rotation where one member of the team is assigned bug duty, and their main responsibility is to fix problems that come up on Meta, monitor the logs, answer sales requests, and anything else that needs to be attended to immediately. You drop everything.

After four weeks, it was my turn.

For the first week, I had the help of another dev while I was tracking which bugs were coming up on the site, and getting a feel for all the monitoring systems we had in place. The week after, I was in the front line. This was a great opportunity to touch many of the dark corners of the codebase I hadn't been exposed to (and quite frankly, a little afraid to touch). It also gave me a chance to get real changes on the site, and in some cases, in rather significant ways. During my down time, I was busy setting up a completely different environment on my Mac that finally came, as well as putting finishing touches on my exercise project. There were a lot of moving parts that week.

Final Thoughts

I've really enjoyed my time here so far - I can't say enough good things about working at Stack Exchange. I think that the on-boarding experience is much better than a lot of other places I've been exposed to, but there are a few things that I think would have been interesting additions:

  • Shadowing or pair programming - I've been a proponent of shadowing and pair programming (for a period) as a method of teaching someone an existing codebase or new technology. This would have been especially useful during my third and fourth week, where I was learning how Stack was organizing their code and what conventions they followed. I think that having someone more experienced with a new developer working on a small feature or bug really allows knowledge to flow quickly. There were a few instances when I did this setting up my environment, but just watching someone code and asking questions along the way might have been a valuable experience. There are just some things that you can't capture in written documentation, chat, or in exploring on your own. Sometimes just watching a veteran work helps a lot.
  • Teaching as learning - I'm also a big fan of using teaching as a way of learning. Of course, there isn't going to be a regular interval of new developers joining my team, but I think that having recently-joined developers help new developers get on their feet is useful for both parties, even for a bit. For one, the experiences are pretty closely aligned: things like setting up your environment for the first time, picking up the nuances of the language, learning your way around the codebase, etc. Chances are, both will have had very similar questions. It also serves as review for the developer that joined before to really make sure she has her bases covered. Think of it something like a final test for the previously minted developer.
  • Documentation and On-Boarding Notes - I generally subscribe to the idea that good code is readable code and maintaining external documentation is cumbersome. For me, it wasn't so much the code itself as it was certain configurations and common environment snags. For example, I discovered that we had custom tools that let me reference static assets in the codebase, but I'd hit an exception if I didn't do so. It's not clear where these instructions should live other than in the minds of other experienced developers, so I would defer to the chat room. But even having a place for common bugs during development would have been useful. What I started doing on my own about halfway through the on-boarding process was that I kept notes of common obstacles I ran into and how to solve them. This was not only be helpful for me when I forgot how to do something simple like migrate an anonymized production database, but I figured it would be helpful for future developers. I think keeping something like a wiki-style tracker or even a single Google doc for these issues would be an interesting experiment.

That's it! Of course, there's still plenty for me to learn in all of these areas, and I'm not yet Jon Skeet by any means. Hopefully, this will give some people insight on what the experience is like joining Stack and help those of you starting off somewhere new as a developer.

Jon

Philosophy is a Practical Major

As someone that pretty recently graduated from college, I inevitably get the question: "Oh, what did you study?" Philosophy, I answer. As someone that works in technology, I also inevitably get the reply, "Oh, interesting."

That second "oh" is hardly ever as enthusiastic as the first one.

I love philosophy, and I'm glad I studied it the entire time I was in college. New York University boasts one of the best philosophy departments in the world (perhaps the very best), and because I was a student of Gallatin within NYU, I got to come up with my own degree program and was able to meld it with the other interests I had.

"Why not computer science?" I am asked. Honestly, at the time, I felt like I knew a lot about programming by the time I went to college and that I would continue to pick up more on my own. In fact, most of the people that I respect and admire as developers were in a similar position as I was. Most of them did go on to become CS majors of course, but I thought all that money for a degree would be better spent on classes that weren't largely review and on something I was just as passionate about. So philosophy it was.

Most people, when they envision what studying philosophy is like, imagine a room full of students sitting in a circle pondering questions such as "isn't everything subjective," and "doesn't saying everything is wrong invalidate that very statement," and other rather pedantic things like that. Now there's plenty of this going on, especially at the introductory levels; however, philosophy any more sophisticated than that eventually becomes extremely difficult. Try to recall the last philosophical debate you might have had with someone - remember how tired you felt after it? It's just very heavy stuff, and to do that with the rigor expected of highly respected philosophers day after day is incredibly exhausting.

What I quickly realized was that philosophy was not so much about acquiring knowledge - I don't think many people really come out of those classes with tangible answers about the world - but rather, it's mental bootcamp. In the same way that you train a muscle, pumping away at highly abstract and complicated topics like ethics, metaphysics, and epistemology make you a stronger thinker. That's how I feel about philosophy: it builds you tangible, fundamental skills in thinking about subjects, even beyond philosophy. I'll touch on just two:

 

Rigorous thinking

This is perhaps the most important skill you get out of a philosophy degree. Much of philosophy is about arguments: how to construct them, how to test them, and how to express them. You spend so much time picking apart the assumptions, logical steps, and outcomes of time-tested arguments of the likes of Descartes, Hume, Locke, Kant, and so many others. In the end, you come out of it having an intuition for detecting fallacy, generalization, and ambiguity.

A typical exercise you'll see in a philosophy course is to compare the contrasting ideas of two different philosophers, and it typically goes like this: (1) explain philosopher A's stance, (2) explain philosopher B's stance and how it contrasts with that of philosopher A, (3) come up with a rebuttal on behalf of philosopher A against philosopher B's original objection. This exercise is extremely useful for a number of reasons, but I'll just take the one I find most important: charity. In philosophy, an important concept in debate is to give the most "charitable" version of the opposing view: that means that you must attack another argument in its strongest form possible, otherwise, that makes your arguments appear weak.

Communication 

Being rigorous in philosophy also forces you to be extremely clear about your arguments, and this requires very strong communication skills, both written and verbal. For example, in order to argue about what the meaning of "meaning" is, it requires you to have a pretty formidable ability to express yourself, and not in a way that is typical for regular conversation (or even this blog). For example, people are not used to how philosophy papers are written. It requires a level of conciseness and clarity you typically do not encounter anywhere else. Take a look at this annotated example of an introductory-level philosophy paper. Like I mentioned earlier, you don't use this language in your every day language (my mother would kill me if I spoke to her like that), but it develops a sensitivity for how one expresses arguments.

As part of another anecdote, philosophy also forces you to be extremely careful about your wording. I've gotten in a number of (probably infuriating) discussions with my friends over the use of a certain word meaning something else than I think they intend to. Many times, philosophical discussions boil down to semantics, and sometimes it seeps into my every day life (to my friends' dismay).

 

This brings me to my final point: that philosophy is practical. What I mean by "practical" is that in the end, I believe that philosophical thinking builds useful skills that go beyond an arcane band of academia. I also do not think that this is unique to philosophy - in fact, I think that many of the liberal arts develop these same skills to varying degrees and that they get a bad rep for being "impractical" as well. Does being a philosopher major qualify you to become an amazing engineer? No, I don't think so. However, I think that being a philosophy major could certainly train you to be a better thinker and communicator, which may certainly make you a much better engineer or fare better in another profession.

Vertical and Horizontal Programmers

I've noticed there are two camps of developers.

I got off the phone yesterday with one of the fine folks at Udacity, easily one of my favorite online education platforms, and I got the opportunity to speak with their director of content development about different approaches to teaching software and web development. What I ended up spending most of my time explaining was a pattern that I noticed among my colleagues and my friends in technology: there seem to be two "ranges" of behavior that developers seem to fall nicely along. I've noticed that a lot of people who have had formal education in computer science (through a higher-education program usually) and those who were self-taught (like myself) tend to exhibit different sets of behaviors. For the sake of clarity, I'll refer to these behaviors as vertical and horizontal behaviors respectively.

Vertical Behaviors: focused on depth and by the book

I should first describe the kind of programmer I notice who usually comes straight out of college. Typically, they have a strong understanding of one or two languages, typically Java or C++ (occasionally Python), and a solid grasp of Big-O analysis, basic data structures, and algorithms. Some of them concentrate in a particular area later on, such as robotics or machine learning, but for the most part, most of these people have similar skill sets and knowledge. Of course, there is a vast degree of proficiency in these areas, but otherwise, their general nature seems to be very similar: focus on performance, efficiency, and the theoretical underpinnings of the code you're writing. There's also a sense of being "by the book" - they tend to look at documentation or formal writings first, they try to fit problems into structures that they are familiar with, and they gravitate towards mastery of a few languages and tools they are used to. They're comfortable with structure, and tend to do very well in larger development shops. Much of this may simply be due to the institutionalization of this kind of programming, but there are always examples of this outside of traditional higher education.

Horizontal Behaviors: a breadth of skill and experimental

The other kind of programmer I've encountered a lot is the self-taught kind. A lot of them entered technology through the web, and picked up a lot of what they know through online tutorials, copying snippets of code, and just mimicking what they've seen others do. They tend to have a relatively shallow understanding of many different technologies - usually front-end web development technologies like HTML, CSS, and JavaScript - and rather than seek mastery of any one of these tools, they depend on offshoots or new tools to improve upon or replace their predecessors. Beyond the basics, this group as a whole has extremely diverse skill sets. Some move on to mobile optimized content, some go into technical library construction, others into completely new territories that haven't been explored. This also has a tendency to leave "gaps:" beyond the rudimentary basics, there is little overlap among them. Partly to do with this diversity is the tendency to want to experiment: they learn by trial and error, they are focused on something that works and overlook understanding about why it will work, and more concerned about shipping code than whether it's "right" or not. Vertical programmers seem to also gravitate towards flatter, smaller shops of development. Again, this may have something to do with how decentralized web development is, but some of the best people I know exhibit these kinds of behaviors have gone through traditional college programs as well.

These have nothing to do with performance or background

Regardless of whether one is more aligned with one type or the other, I have very little to say about which is better. In fact, I'd go so far as to say it has nothing to do with how good of a programmer you are. I've presented two extremes here. I certainly classify myself as a more horizontal programmer than a vertical one, but it seems to me that people are always some mixture of both. It makes less sense to say whether you are vertically or horizontally inclined than to say to what degree you are of both. In many ways, this is similar to whether it's better to be a specialist (similar to what I'm calling vertical) or a generalist (similar to what I'm calling horizontal). The question seems moot. Being a specialist may entail mastery or myopia, just as being a generalist may entail polymathy or lack of focus. They seem to be different styles, but they hardly indicate performance. Take a look at this previous post to read more.

I'd love to hear your thoughts on this.

On Risk

When you’re having trouble making a big decision, think about the worst thing that could realistically happen. If you can recover from that scenario, go for it. If not, back away.

In the space that I'm in, discussions of risk come up a lot. This is an especially hot topic when it comes to students. They get hung up on whether they have the right skillset, the right connections, the right team, and I've started noticing a few patterns. You see, the more I have these discussions, the more I notice that people's fears of what could happen when they take on a project tend to outweigh almost everything else. These fears seem to rest on two things: (1) how possible failure is (the more likely someone is to fail, the less likely they are to endeavor), and (2) the magnitude of their failure (the larger the consequences, the worse).

Why Lack of Confidence Hurts 

First, lack of confidence adversely affects tangible, critical skills necessary to achieve your goals. It distracts. It takes your focus away from the task at hand and unproductively reroutes that focus towards thoughts about perceived incompetence (many times unfounded) and consequences of failure (often exaggerated). With these distractions, you find yourself not performing as well, creating evidence to yourself that you are not doing well, only feeding into this lack of confidence until the cycle ultimately results in complete failure. In effect, you keep shooting yourself in the foot because you're not moving fast enough until you can't move at all. The solution to this is simply dispelling those thoughts, but how can it be done? 

Two Questions 

The best way I've discovered to handle these situations is to be honest with myself about the work and the real consequences involved in the endeavor. Too often, when people are faced with an ambitious project, they reject the opportunity simply for the fact that it is an ambitious project without really considering whether they could do it or what the consequences would be. When I take on a project, I tend to consider both things, and they really translate to two questions that address the two things that I mentioned earlier: possibility of failure and consequences of failure.

  1. Is it really that hard?
  2. What's the worst that could happen? 

For the first question, it's really just a realization that every project is just a collection of smaller tasks. Breaking down those problems and considering each individually is how you gauge your ability to take on the project you're ultimately pulling together. It's much easier to consider large projects as those individual pieces, and oftentimes, it makes the whole thing seem much more achievable.

The second question is the one that I think most people have trouble with. When you're having trouble making a big decision, think about the worst thing that could realistically happen. If you can recover from that scenario, go for it. If not, back away. The key here is to be realistic about this. Failure seems to be something that most people consider absolutely bad in every way, without considering that it's really about the degree to which one can fail based on what the consequences would be. I could fail at something and not have it truly adversely affect me. In many ways, this almost trumps the first question. If there is really no real risk to pursuing your goals, there's not much stopping you. Even if you're not sure you can pull off the individual tasks from the first question, if there aren't severe consequences to it, the only real barrier you have is your time and your willingness to work.

 One last thought

Something that tends to come up a lot, almost to my annoyance at times, is about quantifying risk as a statistic. "Only 10% of all businesses succeed," some say. So they abandon their venture. "This program only has a 10% admission rate," others bring up. So they don't even apply. It is striking to me that many people consider these sorts of things similarly to tossing a coin. It is striking to me that many people ignore that your work, your mindset, and your ability are factors in whether you end up in that successful 10%.

So just try it, especially if you don't really have something to lose.

 

What it's like hitting the front page - twice

Discuss on HN

Yesterday was a crazy day: I hit the front page of Hacker News and Reddit yesterday. It was for a one-hour hack that I built two days before: Bento. It was meant to be a simple reference site for me to direct my friends to when they ask (pretty frequently) the question, "How do I learn to code?"

So this is what happened.

Building It

It was a one-hour hack. The site was really simple, just a bunch of links to sites that I used to learn how to code. I'm pretty much self-taught in development - I only ever took one CS class in college. The list only included things that I thought I knew decently well: HTML, CSS, JS, Python, so on. I also added a way for people to see what they should learn next. It's been done countless times, and I didn't think my iteration would be very different. I even used a boilerplate from another one of my projects (hence the seemingly overbearing technology on the front with angular, bootstrap, etc. for a static site).

I expected it to be like any of my other hacks: build it, share it, maybe it'll get a few hundred views, but then it will die out. I was planning on having a busy day later anyways - I'm writing this post from my room in Anne Arbor because I'm helping out with MHacks so I could learn what to do for the one I'm organizing. So I texted some friends about it, posted it on Facebook and Twitter, and went to bed.

The Next Morning  

I woke up and checked the site. I saw that it got a fairly decent number of likes on Facebook and a few shares on Twitter - a lot more than I usually got, but nothing phenomenal. Just a little over 150 shares combined. I read the comments on one friend's Facebook post, and there were some interesting comments: 

facebook_comments.png

This was the kind of response I was getting in a lot of the channels that it was being shared on. It was great that people were finding it useful. My day at work was crazy, and I was traveling, so I didn't try to make many changes. The biggest shift was moving the project from a free Python dyno on Heroku over to GitHub as a static site. That way, people could make some changes here and there, and I knew there were definitely things I was missing. I pushed it, went to dinner, and went to bed. I had about 250 total shares at this point. Still no big deal - but I was thinking that this could be useful for a lot of people.

The Big Day 

I was gearing up to volunteer at a hackathon this weekend, but I wasn't needed until later that evening. I knew it was going to be a busy day, so I even tried to work more hours earlier that week so I could free up space for Friday. I woke up extra early, did some work, and went to lunch at a BBQ joint I had heard about. I sat at the bar and I declined the beer menu. "It's still pretty early," I had said.

I posted on Hacker News and told some of my friends about it. I ordered my food and kept an eye on the comments. In two minutes I had two points. In five minutes, I had 12. In ten minutes, I was at the bottom of the front page. And in 30 minutes:

555396_10200965109263244_1145830227_n.jpg

Oh wow. All the sudden I was getting a ton of tweets off the share button on the site. I was trying to keep up with thanking people for sharing, replying to comments on HN, texting people that I had made the front page. All this from my phone. The bartender kept looking at me, probably perplexed as to why I was so frantically typing on my phone. At one point, I just set it down - an hour and a half had passed. I looked up at the bartender and asked for the check - there were about five or six pull requests on GitHub I needed a laptop for. But then I thought for a moment and told the bartender:

"You know what, I will take that beer." I had something to celebrate. Front page on HN for the very first time, why not? Today was already a very good day. By the time I left the restaurant, I had 900 total shares. But my day was far from over.

Then Reddit Happened

I went to a close by Starbucks and finally had access to my laptop to handle the pull requests and make some merges. I remember seeing similar links go from HN to Reddit on /r/technology, so I put it there as well. Very quickly, something similar happened:

 

1234431_10200966136608927_1893317308_n.jpg

Then a few hours later, I even hit the home page

top9.png

 More and more of the same. Tweets, more shares, and even some attention from big players

946741_10200966898307969_259948748_n.jpg

At the Hackathon

I still had to go volunteer for the hackathon in the early evening. I knew I was going to be even more busy, since it's the biggest of its kind in the country. Literally thousands of student hackers were there coming from all over the world. I was there to learn how to run a hackathon smoothly, since I'm planning one in November. But I kept getting distracted. Apparently, if you get onto the front page twice, people will suddenly recognize who you are at a huge hackathon (go figure). I was still trying to make more merges, responding to tweets, but like at many hackathons - I was running out of power for my devices. By midnight, I had gone completely dark - the day was pretty much over. 

Take-away

Throughout the day, there were some key things I learned. I made a lot of big mistakes along the way and found out what really helped, and I want to share with you what it is that I got out of it. Here they are:

  • Make your site easy to share. Put your social buttons at the top of your page. Just do it.
  • For your tweet link, include your handle. I can't tell you how useful it was to get people's feedback in realtime as things were unfolding. When there are bugs after a merge, or if the site goes down (it did for a while for me), you'll know.
  • If you're posting on reddit, be careful about using your personal account. Some people advise using a throwaway account for something like this, but I felt I had nothing to hide on my personal account. I'm the moderator of a small API LGBT subreddit (this is not a secret, I've been out since I was 14) and you can imagine there were some hurtful things being said.
  • Don't ask for upvotes. Your content's value should speak for itself. I made the stupid, stupid mistake of asking for upvotes in a rather public way at one point. I got into trouble from the moderators at /r/technology for a few minutes until they understood I wasn't trying to be malicious and fixed it. It's exciting to see something get attention, but don't lose your cool and get carried away. Don't be an idiot like I was.
  • Don't fight back. You are almost guaranteed to get negative feedback and personal attacks. People are going to be mean (regardless of your intentions), and some will even go out of their way to hurt you. Resist the temptation to get into a fight - it's never worth it, especially when they attack you as a person. When it comes to negative feedback, solicit suggestions for improvement, those moments sometimes even turn into opportunities.
  • Thank people for sharing. I spent all day doing this. The most gratifying thing is to see that people enjoy something you've built, and the very least you can do is express your gratitude.

Final Thought 

This morning, after all the madness, I got an interesting question in a comment on Facebook post reflecting on the day: 

question.png

That's about it. I didn't know that this was going happen. There isn't a "secret." If anything, the one thing I think worked was building something people actually liked. There's this idea that if you build something, people will come. Before yesterday, I thought that was mostly a myth - I've talked to too many people in startups and technology to believe that. But after yesterday, I know that it's a little more than just a cliche. Other than that, when it actually starts blowing up, keep a cool head. Just roll with the punches and don't let your head get too big. And thank everyone. If it weren't for their help, I wouldn't have gotten any attention. Thank everyone for sharing. I checked Hacker News again this morning, and remarkably, amazingly - I'm still there on the front page:

Screen Shot 2013-09-21 at 2.28.24 PM.png

Thanks everyone. 

analytics.png

Armies of One

The more that I talk to people involved in startups at my age, the answer to the question, "So what languages do you know?" seems to get more and more complicated. I, like many of my peers, am self-taught in development. We grew up in an age when HTML, CSS and JavaScript were easily tinkered with through personal projects. When Xanga, Myspace, and Geocities were around, those were the hobbies that turned into technical playgrounds. Those that kept with it evolved their skills, and they become extremely sophisticated developers.

What that seems to have led to is a cadre of young people that are more than just academically bred C/C++/Java developers, familiar with Big O and basic data structures. They are kids (quite literally) that have spent years working with front-end, back-end, networking, design on actual projects - what many may call "full stack" developers. Coding is now even easier to learn, the playgrounds bigger and more accessible since my time. I expect only more of this species of developer to evolve. However, when I try and describe this phenomenon to people that are a generation before my own, the advantages of being such a generalist at such a young age becomes opaque. In the minds of many, technology is a specialization, and to do anything more than specialize in a specific part of it is dilution rather than comprehensive understanding. These developers are classified as "jacks of all trades, masters of none." It then occurred to me: what in the world do I call myself, if nothing but a jack?

Now don't get me wrong, I'm not saying that this generation of developers are all technical gurus, but it's sad to see that their precociousness be whittled down to distracted developer, simply because we don't have a term. I remember having a particularly memorable conversation about the generalist vs. specialist, and the jack of all trades accusation inevitably came up. I retorted:

Armies of one. The term has stuck in my mind ever since.

Autodidactic, polymathic - to varying degrees. But what is important is that they possess a technical acumen that is composed of a large set of moving parts, rather than a specialization in a narrow band. Specialization is treated as myopic rather than advantageous. Generalization is treated as flexibility rather than evidence of uncommitted energy. These are people that are capable of building their own massive projects (that's how they became so capable in the first place), and they are attracted to others that are the same.

I run into many of these AOOs as someone deeply involved in NYC's student entrepreneurship scene for several years. They are rare, but not as uncommon as one might expect. I wrote this to provide a different perspective on a generation of developers that requires a more fitting description. Maybe it will stick with you too.

Social Networking is Like Fast Food

Discuss on HN

I've been thinking about social technologies for a very long time, and I think that people are starting to get the sense that there's something wrong with social networking. Facebook, Twitter, LinkedIn, and the like - they are social in so far as these technologies are about people, but what they do for people is only questionably so. I tell people this, and they go, "What are you talking about? I have X number of people connected to me all the time! I'm more connected than ever!" But does that mean they are actually more "connected?" Do you really feel close to the 1500 "friends" you have on Facebook? I don't really think so.

I think of social networking right now a lot like I think of fast food.

Think about the experience of going into a McDonald's. You go in, order a Big Mac, for rather cheap and for something quick. First of all, there's no cooking done in a McDonald's - you won't see any pots or pans back there. There's no denying that what you're getting is not made the same way you would have it at home - it's precooked and being reheated. So you get your Big Mac, it certainly looks nutritious, it certainly tastes good, but you're not getting much more than empty calories. I can't tell you the number of times I've eaten at McDonald's and still found myself hungry afterwards. Also, you don't eat at the McDonald's. You tend to take it home or back to work, and you eat it alone. In the end, I came in for something quick, cheap, and tasty - I got those things, but I don't feel good about it afterwards.

Contrast that with going to a local restaurant you like.

Someone crafted your food - it was cooked for you. There's usually history and culture and tradition about that food. The staff is friendly and you're being catered to. They have stories to share with you, should you ask. You also tend to go with friends to these sorts of places. You have conversations with them, and reminisce, and bond over the things you're eating. These are things you remember. In the end, you go home thinking you had a good time, and you should go back to that restaurant some time soon. Most of all, you feel closer to the people you went with.

Facebook is like McDonalds. I am getting things that are packaged to look and feel social. They are provided to me quickly and cheaply. But I hardly ever feel good about going afterwards. Status updates, photos, links to articles, they are technically social things from my "friends," but they are nothing like the real thing.

Tweets are the chicken nuggets of social networking.

I don't remember all the times I go to McDonalds. I do remember the times I went to that bistro with my friends, even if the food wasn't phenomenal. In the same way, when I look back at my life, I won't remember the tweets I sent, the statuses I posted, the photos I put on Facebook, but I will remember the times I went to a bar crawl with my coworkers, or the first time I met each of my friends in a new city, or the time I cooked Thanksgiving dinner away from home with them. Social technologies should be doing more to create experiences, rather than log or share them in artifacts as text, photos, or links.

Technology can do so much more for our social lives than it is now. Where's the restaurant equivalent of Facebook? Why haven't we built it yet?

As developers, I think we can. In fact, I think we should. It's been the problem I've been trying to solve, as I know a lot of people and startups are too. I'm not saying we should destroy social networking in its present state, in the exact same way I don't think we need to kill fast food. Fast food has helped feed a lot of people that wouldn't otherwise be fed, for cheap. I also can't begin to tell you how gratifying it is to eat chicken nuggets sometimes. But I don't think the world should be dominated by fast food, for our long-term prospects. Similarly, I think we can keep the current social tech, and we may always need it and it may never go away because it does good things for a lot of people. But what happens when I have 10,000 followers, and no one to grab a quick burger with?

I just think we can build something so much better.

SLOPE: A proposal for better time representation

My first post and something I've been thinking about for a while.

Discuss on HN

The problem with time data

I work a lot with times. As someone who’s been tackling the events space and especially with calendars for a while now, dealing with times is something that I have to deal with constantly—and with great difficulty. Understanding calendar formats, RFC specifications, different date objects in different languages (think JavaScript Date() nightmares)…these are all things that make my life as a programmer handling times an absolute mess. The problem is that there is no universal and “complete” (more on this) standard for representing time that is computably friendly. Here are just some of the issues:

  • Most standards for time representation are (what I call) “incomplete.” A “complete” representation for time should be a singular representation that includes all and only the essential information for any point, interval of time, or recurrence of time intervals independent of geographic considerations (time zones).
  • Time objects are hard to specify as a single point of time, an interval of time, or even recurrences of time intervals. You commonly need separate representations for each. They can and should be one representation.
  • Time zones. Oh god, again. Time zones.
  • Extremely disparate standards for easy data manipulation and formatting. It is currently a free-for-all when it comes to representing times: YYYY:MM:DD THH:MM:SSZ or Unix Timestamp? What about intervals of time? Why not MM/DD/YYYY?
  • The representation should account for simple recurrences of points in time or intervals (every Friday at 2:00 PM, every other Tuesday at noon for an hour). They should be intuitive to understand.
  • The representation should be easily transferable between platforms. Ideally in a text format.
  • Current representations (most notably according to RFC 3339 and the like) are human readable, but painstakingly difficult to calculate with.

Introducing SLOPE

As I was tackling these issues, I came up with my own solution. SLOPE is an acronym for the necessary parts of the representation: start, length, offset, period, and end. I’ll explain each, and go through with some examples. SLOPE is definitely not in its final form (take a look at the Pros + Cons section later), and any suggestions would be helpful.

  • Start – an integer, the start of a time interval as a Unix timestamp format according to UTC (never a relative timestamp)
  • Length – an integer, the number of seconds since start of the interval. Single points in time should leave the length as 0, the default value
  • Offsets – an array of integers, ideally organized in increasing order, that specifies the number of seconds since the start that each recurrence should begin with.
  • Period – an integer, the number of seconds between each occurrence’s beginning timestamps (including the original interval’s, specified in start) for each occurrence
  • End – an integer, a Unix timestamp that specifies when all intervals in all recurrences should stop. Intervals that include the end timestamp within it are also considered out of scope

To represent all of my birthdays (the full length of my birthday - May 15th - from midnight to the next midnight) up to but not including the year 2050 starting in 1990:

{
start: 642758400,
length: 86400,
offsets: [],
period: 32536000,
end: 2536218000
}

Let’s go through the example. The first thing to notice is that the representation is in JSON format. Why JSON? Quite simply, as text, it’s easy to transfer between platforms through AJAX—and it’s what I used for my own projects. In addition, because the offset is sometimes represented as an array (more on this later), finding a text-based representation that could accommodate complex objects naturally led to JSON. However, this is not a strict specification. As long as the elements (start, length, offset, period, end) are represented in order, a SLOPE representation could be comma separated or even a multidimensional array.

The next thing to notice is start. This is a pretty straightforward Unix timestamp according to Coordinated Universal Time (the number of seconds elapsed since January 1, 1970 GMT). Just regular Unix timestamps, if you want to represent a point of time before UTC, use a negative integer. In the example above, 642758400 is “May 15, 1990 at 12:00 AM PST,” my birthday. The important thing to note here is that start specifies the beginning point of what I’m calling the template interval of the SLOPE. More on this later.

After the first field, we have length. This is also pretty straightforward and is just the number of seconds since the start that the template interval is supposed to last. In the example above, 86400 is the number of seconds in a day (24 * 60 * 60). If I wanted to only represent the beginning of my birthday – a single point in time – I would leave length as 0, the default value

Recurrences

So far so good, but here’s where it gets interesting with recurrences. Remember that I originally created the SLOPE standard representation to help with calendar events. Representing something like a recurring lunch with a friend, say “every Monday at noon EST for an hour,” was a very difficult thing to do. The last three parts of SLOPE – offsets, period, and end – are specifically used for recurrences. They are all used with the template interval – the interval of time specified using start and length – as a starting point. All other intervals of time based off of the template interval I’ve called repeating intervals. In the example above, “May 15, 1990 12:00:00 AM PST for a day” is the template interval, while all other birthdays after that (for 1991, 1992…2050) are known as repeating intervals. This is very technical I realize, but will make explaining offset, period, and end much easier.

I’m going to skip offsets for now and go to period first (you’ll see why). Simply an integer, period is the number of seconds between each repeating intervals’ start times beginning with start. In the example above 32536000 is the number of seconds in a year (365 * 24 * 60 * 60), since I want every repeating interval to start a year after the previous one. If I want to find out when my first birthday is after I was born (in 1991), I simply add start and period together (674294400=642758400+32536000). Under SLOPE, each repeating interval will have the same length. Period only specifies

Let’s now deal with offset, probably the most difficult part of the SLOPE representation to grasp. This field is used to create complicated recurrences that take the original recurrence according to start, length and period and duplicates the original recurrence for each value in the array. Each value within the array is another Unix timestamp that each new recurrence should use instead of start. If there are no new recurrences, the array is left empty. In more simple terms, imagine taking the original recurrence according to start, length, and period, duplicate it, and shift the entire recurrence using the a new start entry. This is best explained with the example we have. Take the SLOPE we have now, representing every birthday I have for the length of the day every year from 1990 to 2050. I could also represent my sister Tiffany’s birthdays (beginning with July 21, 1991 12:00:00 AM PST or 680083200 in Unix) in the same SLOPE, simply by specifying adding her birthday as a value in the array. It would take the entire recurrence I originally had and duplicate it but with a “new start.” The revised SLOPE would look like this:

{
start: 642758400,
length: 86400,
offsets: [680083200],
period: 32536000,
end: 2536218000
}

I could do the same to account for my other sister Eileen’s birthdays as well (starting September 8, 1993 at midnight or 747475200), to accommodate all three of our birthdays in the same SLOPE:

{
start: 642758400,
length: 86400,
offsets: [680083200, 747475200],
period: 32536000,
end: 2536218000
}

Offsets should never be before the start for any reason. If there is a timestamp that is earlier than one in offsets the earliest value should be switched with start (more on this later). In addition, it is best practice to order the timestamps in offsets in ascending order from earliest to latest.

Finally, we reach the last part of the SLOPE representation, end. This is a Unix timestamp that specifies a ceiling for which all repeating intervals must end. In the original example, I used the end timestamp corresponding to May 15, 2050 12:00:00 AM PST. In this example, the SLOPE would not represent my birthday for 2050, because it is beyond the “hard stop” specified in end. Only if end were May 16, 2050 12:00:00 AM PST (exactly a day after, or the exact amount length is after that interval) would my 2050 birthday be included. This would also be the case for any recurrences denoted with offsets.

Pros + Cons

Pros

  • Completeness - I made SLOPE so that it could be unified representation for times for most use cases: single points in time, intervals of time, recurring intervals of time, and even multiple recurrences with offsets. These are all encompassed by a single representation that is not accounted for in others.
  • Better computability - SLOPE makes computation for times (expanding and calculating times) much more obvious than RFC3339 representations, especially when it comes to recurrences with things like RRULE, &c. RFC3339 and recurrence rules in current calendar formats are unintuitive. It strips away the need to convert RFC3339 formats to some computable format and back as well as the need to understand the arcane intricacies of recurrence rules. You just need some basic math.
  • Geographically independent - There is no reliance on time zones. I created SLOPE with the intention that it would be accurate regardless of its implementation in a specific region. Time zones embedded into time representations - in my experience - have made standardization and accuracy nightmare.
  • Intuitive - I did everything to make SLOPE intuitive. The name "SLOPE" is meant to be an easy-to-remember acronym that accurately explains a complete representation of time. The concepts explained by each part of the acronym are meant to be obvious as well.

Cons

  • Leap years - The biggest problem and probably a breaking point for the standard. The problem with unix timestamps is that it cannot account for leap years. This puts a big snag in the road for the concept of easily computable dates. My intuition says that the onus for accounting for leap years falls on the developer, rather than on the standard itself.
  • Not directly human readable - Though conceptually easy to understand, the reliance on Unix timestamps makes it difficult to hard-code a specific time without some assistance. However, this is something easily done with conversions built into most if not all major programming languages.
  • Lack of recurrence exceptions - One of things SLOPE fails to capture is recurrence exceptions. Returning to the example of my birthdays from 1990 to 2050, if I wanted to exclude my 20th birthday, I wouldn't be able to like some other representations could. Then again, exceptions may be just that from a development standpoint - exceptions: I see them more as an accessory to time representation (like timezones) rather than a necessary component.
  • Verbose - I do think that SLOPE can be verbose in a number of situations, especially when talking about single points in time or single intervals or time. Perhaps a short hand for SLOPE could be used (remember JSON is not a strict guideline) to solve this issue:

General Format:
start;length;offset1,offset2,...,offset3;period;end

Single point in time:
12345

Single interval in time:
12345;60

Daily recurrence, no offsets:
12345;60;;86400;50000

Daily recurrence with offsets:
12345;60;23456,34567;86400;50000

Your Thoughts

Though I've spent a lot of time thinking about this, I certainly haven't thought up of everything. There must be things that I've overlooked, so I'm looking to your comments about the proposal. In the end, I wanted to make things easier for myself when working with times, especially when it came to calendars. I think that SLOPE could have some interesting potential with some new insight.