Prior to working at RJMetrics, I worked at Wells Fargo Bank. I had positive experiences there and learned a lot, but I wanted more. I made a list of attributes that the optimal job would have for me and found a match with RJMetrics. I am loving my experience and I have tried to pick out the top three reasons others working in a more traditional job should consider working at a startup.
1. Rapid growth and how it will impact you and the company
As a startup, one of our main goals is to grow constantly and exponentially. It takes a wide range of skills that span across many disciplines and departments to successfully achieve the goals of a company. RJMetrics aims to achieve its goals with a small, dedicated team. I have found that my learning curve here is incredibly steep because I want to get better every day. I know that if I get better, there is a higher probability that we succeed and I know that all of my team members feel the same way.
I have been working here for a little over five months and have been involved in projects that span across marketing, sales, law, technology development, product management, UI design, SEO, social media, client relationship management, and the list goes on and on. Sometimes my role is very client facing and at other times it focuses on our internal needs. The range of skills I develop here is much broader than any previous job. It is an opportunity to reach my full potential, uncover talents that I didn’t know that I had and then strive to reach my full potential in those areas as well. I can’t think of a more fulfilling experience when it comes to an employment opportunity than the growth of the company and the expansion of your own personal skillset growing together.
2. Know what you are doing and why you are doing it
I understand our value and I am aligned with our vision. I didn’t walk in on day one with the full understanding but through my involvement in a broad range of projects, these things became apparent quickly. I believe in what we are doing, and this allows me to play a part in determining what my contributions should be and to accurately evaluate what kind of impact they are having. This level of understanding and transparency coupled with the ability and freedom to expand my skillset as needed makes me a highly effective team member. It makes us all highly effective.
It is obvious on a day-to-day basis that we are all working towards the same goals, and this establishes a high level of trust between us. I am much more willing to take the time out to share something with another team member because I know that that person is going to use that information to make a valuable contribution. The company is willing to invest in me because they know it is a sound investment, and in turn I am willing to invest more into the company. This makes us capable of achieving what it would take a much larger team to achieve whose members were not aligned with the vision of the company.
3. Constant communication
Communication is key and I have never been in a professional setting where it was as easy to do it as it is here. No red tape. No bureaucracy. Nobody is out of reach. Whether it be a complex question, a product improvement suggestion, a process recommendation, telling a joke or just yapping about something you find interesting everyone is accessible and ready to listen. The exchange of ideas, knowledge and insights is so easy that you can come in with an idea in the morning and be executing by the afternoon, having already vetted it with your team members. We are empowered by the ability to gain the knowledge that we need quickly and to put it to use to solve problems, to innovate and to become more efficient.
I give kudos to my colleagues here for putting together a team of talented individuals who have found common ground in wanting to see their company succeed and in their own continuous development professionally and as individuals. We are lucky in that RJMetrics is allowing us to achieve all of these goals simultaneously. If you think this is something you want to try out, you can learn a little more about us, a little more about our recent funding, why we call Philly our home, or check out our available positions.
Perhaps my experience is unique, I would be interested in hearing others’ experiences whether they be good or bad so feel free to comment away.
I recently spent some time digging into the PHPUnit mocking framework to clarify my own understanding and better decide how we can improve our unit testing tools. I've found the PHPUnit documentation on the subject to be inadequate for anything but the most basic use cases. (The full API documentation does not include the mocking framework, as far as I can tell.) I'll try to clarify the one aspect of this that confused me the most and caused me to lose the most time.
Stubs vs. Mocks
This is touched on in the documentation, but I'm not a fan of the way it's described. "Mocks" are objects which have some of their methods replaced by stubs (methods with no behavior). "Stubs" are objects which have stubs for all methods, with expectations optionally placed on certain methods. As far as I can tell, the distinction between a mock and a stub comes when you provide a list of explicit methods to stub, or not. When you provide a list of methods like so:
$mock will defer to the original class's methods for all method calls except for "methodOne" and "methodTwo". These methods will always result in successful calls, but will not return any values.
When you don't provide any methods, you get a stub with no functionality. This object has the ability to act in place of the mocked class wherever it is used, but has no functionality on its own. Once you create this object, you can add expectations and return values to force it to behave in ways that are useful to you:
This strategy allows you to build piecemeal mocks that build functionality you need as you go. I find this the most useful way to use PHP object mocking. If you're creating objects which need to be "partially mocked" in order to test, you might want to reevaluate the structure of your objects--possibly moving those partially mocked methods into another component, which can then be fully stubbed.
At RJMetrics, I've been experimenting with a mock dependency container which will provide stubs for all our objects. This will give us the greatest flexibility in terms of being able to add units of functionality needed for a specific test, while still maintaining internal consistency with all of our dependent objects.
Here at RJMetrics we recently added 10% time to our development cycle. I decided to do my project in clojure because I needed easy concurrency and I wanted to learn a new skill. Since I have never worked in a lisp-based language before, I found the fact that clojure has two ways to resolve a symbol's scope to be a little confusing at first. I found many posts that helped explain the difference, but few took a beginner approach. I hope this post will clear up any confusion that new clojurians may have.
Background
A "scope" in computer science is a context that is created to encapsulate the values of variables (or symbols in clojure) to smaller pieces of code. Without any scopes, every variable would be global. Can you imagine if every time you used the variable x or i you had to worry about interfering with another piece of code? Luckily, most sane languages provide a means of creating and restricting the scope of variables. Some languages (eg. Clojure, Common Lisp, Perl) have two means of controlling scope, lexically (aka static) and dynamically.
Clojure Terminology
These are a few basic clojure terms you will need to know to understand the rest of this post.
- A symbol is similar to an identifier in other languages. Symbols are used to associate values with common names. In an imperative language like PHP you might say something like "$x = 5;". Now the variable $x equals 5, in clojure semantics I would say the symbol x is associated with the value 5. Clojure has an opinionated philosophy on separating identities and values.
- A value is just actual data, such as a number, string, function (code is data), vector, object, etc..
- A var is a way of tying a symbol to a potenitally changing value; they are created using the def macro. This will be covered in more depth when I cover dynamic scope.
Lexical Scoping
The lexical scope of a symbol is the "textual region" (aka code block) where the symbol definition occurs, plus all child code blocks. You can imagine your code creating a tree of code blocks that the compiler can traverse. What defines a code block is dependent on the language. Lexical scoping is by far the most common amongst all modern ALGOL-influenced languages (C, Java, etc). When the compiler is trying to resolve a reference to a variable, it will first look in the inner-most scope and continue up the levels of scope (code blocks) until it finds its declaration. Lexical scope is sometimes called static scope since all symbol references can be resolved at compile-time, as opposed to dynamic scope which needs runtime information. In this C example below the variable i is available within the for and while loops, because the while loop is below the for loop. However, i is not available outside of the for loop since that is above the scope in which the variable was defined.
This is the scope you will be using in clojure the vast majority of the time. The main difference is the scope blocks are declared explicitly by the programmer in clojure rather than by the syntax of the language. In clojure, a lexical scope region is defined by a let block. In the code example below, x, y, and add are all symbols that are associated with the values 3, 5, and a function that adds all of its arguments, respectively (remember functions are first-class values).
The lexical environment is encapsulated by the parentheses (aka parens) surrounding the let block, which means as soon as I close the let statement, those symbols go back to their original values. Here's a more complex example:
The def statement on the first line creates a new global var with the value 2 and associates it with the symbol z, which is similar to how constants would be used in other languages. In this example, we defined a new let block within the previous one. Remember, when the compiler tries to resolve a symbol it will recursively go up each scope block until it finds the first symbol definition that matches. In the inner let block we used the symbol z, which already had a global definition. In this case, z is being shadowed by the value 10. As soon as we exit the let block, the value of z will return to its global definition. So by using let blocks to declare your local symbols, you never have to worry about overwriting some global value like you would in PHP or JavaScript. One thing to remember about lexical scoping is that it only deals with the text inside of the let block. So even though I called add-3-to-z while the local value of z is 10, the text defining the function add-3-to-z is outside of the let block so it defers to the global value of z.
In this last example, while inside a let block, I create and return a function that takes 1 argument and adds it to the value of x. Since the code that created the function was within a let block, the value of x is 1 even if the function is called outside of the let. Due to the fact this function retains its scope at the time of creation, it is said to "close" over the lexical environment. This type of function can be referred to as a lexical closure, or just closure for short (this is where clojure gets its name).
Dynamic Scope
Dynamic scope is another method of resolving symbol names. A dynamic scope environment is created using the binding macro, which syntactically looks like let, but has some big differences:
- You can't create new symbols. Every symbol has to be associated with a previously initialized var.
- The initialized vars also have to be defined as "dynamic".
- The new bindings affect the entire call stack, not just the immediate text region.
- The bindings that are created are pushed onto a thread-local stack and cannot be shared between multiple threads.
We can't make up new symbols, so we have to shadow existing vars. This seems like a restricted version of let initially, but after a few examples the purpose of dynamic scope will hopefully be made clear. We already know we can make new vars using def, which, as I mentioned before, act like constants. The exception to that rule is when they are defined as dynamic. Functions defined with defn are stored in a var, and so they can also be declared as dynamic (the defn macro uses def internally). A var can be defined as dynamic simply by adding ":dynamic true" to its metadata. Let's take a look at this next example:
[NOTE: Before clojure 1.3 all vars were implicitly dynamic, which means there is a lot of broken code on the internet (including the official clojure website at the time of this writing)]
The vars x, y, and sum-of-squares are all declared to be dynamic at the top of the file. The function sum-of-squares-for-x-and-y calculates the sum of squares for the vars associated with the symbols x and y. Remember x, y, and sum-of-squares refer to vars and not values, the value of the var is looked up every time this function is run. When the clojure runtime encounters a var, it first checks for any thread-local bindings, if it can't find any it will return the root value of the var which is shared by all threads. So in the above example when we first call sum-of-squares-for-x-and-y, all of the vars referenced in the function definition defer to their root binding. Next, we push new bindings on to our thread-local stack and call the same function, and now the function uses these new values and returns a different result. The key point here is the entire call stack below the binding point is affected, not just the text within the binding block. Once we end the binding block, the values get popped off the stack and return to their root value.
In the above example, we have a function that returns an adder function for the var x just like we did earlier using a let block. Since x is a var, the value is determined at the time the function is run, not when it is created. You cannot close over a dynamic scope! In the example below I got around this by lexically shadowing the var x with its current dynamic value.
So why would you ever want to use dynamic scope over lexical? Let's take a look at how dynamic vars are being used in the core clojure library. The stdin and stdout file handles, which many core I/O functions use (eg. println, read), are defined by the dynamic vars *in* and *out* (by naming convention, dynamic vars are wrapped in asterisks (aka earmuffs)). Maybe you want your application to print its output to some log file instead of stdout. We can simply dynamically re-bind the *out* var at the base of our application.
Now the println function (and any other function that uses *out*) will print to log.txt. Let's try something a little more complicated. Say we have a multi-threaded application and we don't want the print output from each thread to be intertwined with each other. One way we could accomplish this is to have each thread print to its own log file. Since each thread maintains its own stack of bindings, this is trivial to do.
Now each threads' print statements will go in to their own files. Note that it is rare to directly make java threads like this in clojure since clojure already has more expressive concurrency primitives (agents, futures, pmap, etc). However, since those all use threads internally these same binding rules apply which can lead to unexpected results. Some other use cases of dynamic scope include the following:
- You can debug function calls by dynamically overriding that function to print every time it gets called. Checkout clojure/tools.trace for the excellent dotrace macro.
- The clojure/java.jdbc library uses a dynamic var *db* so that you don't have to pass in a database connection to every function call. You can just use the with-connection macro to bind *db* to the database you want to use, and then perform any database queries within the body of the macro.
- It allows you to mock functions that have side-effects to make them more unit-testable.
- If you look at the clojure core library you will find a lot of other use cases. For example, the current namespace is stored in *ns* and you can use the in-ns macro to temporarily bind that to some other namespace and dynamically add functions to the newly bound namespace.
Conclusion
In general, when you think you need a dynamic var you probably don't. Lexical scope is easier to understand because the programmer can determine the resolution of the symbols by looking at the code. Using dynamic vars also breaks referential transparency, which greatly reduces code readability and testability. However, clojure is a practical language that allows the programmer to come up with creative and expressive solutions to problems, and recognizes that sometimes using dynamic scope might be the most straightforward way to solve a problem.
As programmers, we create code and libraries for others to use. There's no getting around this fact: if nobody else ever reads your code, you probably aren't going to get very far. This gives rise to a question: how can we tell if the code we've designed is useful for others, and also ourselves? The classic text Structure and Interpretation of Computer Programs (http://mitpress.mit.edu/sicp/) spends a lot of time delving into this issue. The first section of Chapter 4 contains a few sentences that I can safely say have had a bigger impact on how I write code than anything I've ever encountered. The text reads:
Metalinguistic abstraction -- establishing new languages -- plays an important role in all branches of engineering design. It is particularly important to computer programming, because in programming not only can we formulate new languages but we can also implement these languages by constructing evaluators. An evaluator (or interpreter) for a programming language is a procedure that, when applied to an expression of the language, performs the actions required to evaluate that expression.
It is no exaggeration to regard this as the most fundamental idea in programming:
The evaluator, which determines the meaning of expressions in a programming language, is just another program.
To appreciate this point is to change our images of ourselves as programmers. We come to see ourselves as designers of languages, rather than only users of languages designed by others.
Admittedly, this is very general and abstract! Your eyes might have glazed over at the phrase "metalinguistic abstraction", but try to bear with me. How can we take this idea and use it in our day-to-day work? The next paragraph offers an example from earlier in the text:
In fact, we can regard almost any program as the evaluator for some language. For instance, the polynomial manipulation system of section 2.5.3 embodies the rules of polynomial arithmetic and implements them in terms of operations on list-structured data.
Ah, so what they're saying can apply to any program you write! This takes a shift in mindset to appreciate, but once you make that shift, you'll find that your programs are clearer and more helpful. If you follow these principles, your code will give you the ability to:
- Work with the code instead of having it work against you.
- Solve problems in new and interesting ways that you couldn't fathom when you did your initial design work.
How does this apply to your domain?
You might be thinking: "This is fine for somebody creating small systems that are obviously reusable, like mathematics packages. However, my application in <your domain here> doesn't facilitate that kind of reusable design!" Let's consider a practical example: the RJMetrics application. This is a large system created for gathering data and performing analytics on it to provide useful business knowledge to our clients.
I'm going to propose a different separation of concerns. Instead of only creating an application for people to use and analyze their business data, we should design a language for solving business analytics problems, and then build an application on that foundation. You can do this for any domain--of course, the previous sentence can always be rewritten to include your problem domain instead of business analytics.
An example will probably help to clarify what I mean. One of the fundamental building blocks of our system is the "structure" of a client's database. Assume, for the sake of this example, that this system only supports relational databases (though this is not the case for the RJMetrics platform). This is made up of your standard database objects: tables, columns, schemata, etc. Now, what happens when you decide you no longer want to support only relational databases? You want to move into the NoSQL realm, so you need ways of representing the structure of a NoSQL store that can be used in the same fashion as your current structure building blocks. We should be able to assemble the existing pieces (or introduce new ones transparently) so that NoSQL data stores can be manipulated and used in the same fashion--you certainly don't want to reimplement your fundamental building blocks to accomodate a new way of structuring data. If this is an easy task (and you'll know when you go to implement it if that's the case!), you might have designed your system in a language-oriented way.
Note I'm using the word "language" in a different sense than we usually mean when discussing code: I don't mean a programming language, and I'm certainly not advocating that you implement a new programming language for your specific problem domain! Rather, I'm talking about the API you use to build the application--and if you consider these API design principles, it will be much harder for you to end up with a poorly-designed interface.
How can you tell if your platform has these qualities?
This is a difficult problem, but here are some thought exercises to make it easier. Try considering this: assume that tomorrow, you're leaving the application business and want to focus solely on creating a platform for others to use and create their own solutions. First, would people be able to comprehend the language you've provided for them? Do the primitives and means of combination make sense? Second, if somebody comes up with a new way of using your platform, are your current tools going to work with their new ideas, or are they going to break? On a more implementation-specific level, any side effects in the code will make this impossible--someone will inevitably call a function with side effects at the wrong time and it will blow up in their face. Functional solutions are the only option here.
Or, to put it more succinctly:
- Could you open source your platform tomorrow?
- Would you be embarrassed by what people saw?
The RJMetrics codebase isn't there yet (personally, I've seen very few production codebases that are), but as we consistently refactor and notice patterns, the language evolves and becomes more useful. A nice side effect is that these changes stack on top of each other: every incremental change makes it easier and easier to improve it even further.
On that note, this is the most difficult part of getting to this point with your code: you can never neglect refactoring. No codebase gets it right the first time, so you have to be willing to find patterns and exploit them. That's the only way you can ever evolve your platform to the level of a reusable language.
Posted by
Bill Piel on Wed, Jan 04, 2012 @ 07:00 AM
Here at RJMetrics, we like to experiment -- especially with ideas that have proven successful for other companies. For this reason, like many startups before us, we have adopted Google's 20% time policy. Huzzah! Because this is an experiment, we defined a fixed trial period of three months. We also cut it down from 20% to 10%. This fits well with our 10 day sprint cycles, and seemed like a good way to get started. After three months, we will assess the results and decide where to go from there.
What is 20% time?
In case you haven't heard:
Google engineers are encouraged to spend 20% of their work time on projects that interest them. Some of Google's newer services, such as Gmail, Google News, Orkut, and AdSense originated from these independent endeavors. (Wikipedia)
Although big, amazing things are always a possibility, I don't expect any of us to develop the next Gmail (RJmail?). I do expect lots of little, amazing things to come from this, such as small improvements and optimizations of our code and simple utilities that solve daily annoyances.
What did I do with my 10% time?
I experimented with implementing a general lazy loading solution in PHP for properties of model objects that reference other objects. Nothing I did was groundbreaking. But these are the sort of optimizations that can easily be overlooked in a fast-paced development environment where there are many competing priorities. That is, until 10% time comes along with a respite from the team todo list.
What's the problem?
As you might expect, our codebase contains many class definitions for model objects, and by that I mean, objects whose primary purpose is to hold data and map roughly to a record in a database table. Given our business, an obvious example is the Chart object. It has some simple scalar properties such as name, time range and color scheme. But Charts also have relationships to other models, such as:
- the user who owns the chart
- the client whose data is displayed by the chart
- the trend that provides the data
The question is: does the Chart object have a property that directly references each of these related model objects? And if so, does that mean we must load and construct each of these objects in order to build a Chart object? These questions are answered by the implementation details of our Data Access Objects (DAO, pronounced “Day-O”, like in Harry Belafonte's Banana Boat song).
Data Access Objects (or “mappers”, as we affectionately call them at RJMetrics) handle the process of constructing model objects from the data in our database, and also saving the data back to the database. If one model object relates to another we might load them at the same time, which allows us to do this:
But, this can be expensive. Imagine we want to display a list of Charts. We might be loading many, many Charts and only using the 'name' property for each. In this scenario, also loading up each of the user, client and trend model objects is unnecessary and may introduce undesirable delays.
If we have implemented our mappers to not load related objects with the Chart object initially, but we are in a situation where we do need these objects, we might end up with code like this:
Look at all those mappers! What a mess. The previous code example is much cleaner. Can I please have my cake and eat it too? Probably!
There are corresponding changes that need to be made both in the model objects and mappers, but the core of this solution is stupid simple. Here's the PHP:
What is going on here? Memoization! I dislike saying this word almost as much as I love the concept. The above example defines a class that encapsulates a single property which is either a value or a closure that provides a value. If it is a closure, the first time the LazyLoader is invoked, the result is memoized, stored in memory for later reuse. The closure is discarded and never called again. This is fine as long as we don't expect updates to the database record to automatically propagate to this property.
How do we use it?
To enable lazy loading for a property, we must make some simple changes to the model object and corresponding mapper. Here's an absurdly simple example:
Above we have a simplified Chart class with two public properties, “name” and “user”, with getters and setters for each. This demonstrates the difference between a typical property implementation and a lazy one. Instead of storing a value, we store a LazyLoader object and it handles the details of memoization for us.
The other end of our modifications lie in the DAO, the ChartMapper class.
We've hidden away the call to the userMapper in the closure that is passed to the LazyLoader by the Chart object. Now this object is loaded on demand without having to mess with additional mappers in the calling code. Success! We can enjoy eating our cake that we still have.
Things get more fun when you realize the power of stringing lazily-loaded property calls together.
The line of code above might be used to get a list of all the users a given chart could be shared with. Incidental complexity has been stripped away. Who knows how many mappers are involved in the line above? Not me. That's a detail that need not concern me as long as the line is functioning and performing well.
Today, we are proud to announce that RJMetrics has closed a $1.2 Million financing round from a syndicate of world-class technology investors. Our new investors include
SoftTech VC,
Lerer Ventures,
SV Angel,
Zelkova Ventures,
Upstage Ventures,
Red Swan Ventures, Venture Capitalist Jon Anderson, Wharton Professor Kartik Hosanagar,
DuckDuckGo CEO
Gabriel Weinberg, and
Fab.com CEO
Jason Goldberg. Jason Goldberg has also joined our Board of Directors.
As a fully-bootstrapped and profitable company, the decision to raise capital was not an obvious one. Jake and I both worked as VCs prior to founding RJMetrics, and that experience taught us the benefits of waiting as long as possible to raise capital. We agreed from day-one that we would only take on investors when we were certain that it would generate positive returns.
I’m happy to share why that day finally came.
Why We Did It
Opportunity. In 2009, Jake and I emerged from my attic with a minimum viable prototype of our hosted business intelligence product. Jake hammered the phones and we got our first few customers. By 2010, we had enough revenue to make our first hire. We improved our processes and product by leaps and bounds. Word started to spread and more customers arrived.
By the end of 2010, we had grown the team to four people and moved into a real office in Center City Philadelphia. In 2011, our customer count exploded, our headcount tripled, and our product got even better.
We got efficient. We added automated marketing with Hubspot, CRM with SalesForce.com, customer support management with Zendesk, code control with Github, product management with FogBugz, and (of course) robust analytics with RJMetrics.
Thanks to our own product, we were seeing compelling data about customer lifetime value and retention. This led to the conclusion that the only thing between us and even faster growth was more rapidly iterating on our product and getting in front of even more people. Raising money was the key to going after this this huge opportunity as aggressively as possible.
Amazing Investors. We knew that the right investors could contribute far more than capital. We spoke to a number of institutional investors and angels, and ultimately decided on an angel syndicate because of the speed, flexibility, terms, value-add, and diversity they were able to provide.
We were fortunate to be oversubscribed almost immediately. This gave us an opportunity to be selective about the partners we chose. Naturally, everyone is well-connected, smart, and experienced. If you look closely, however, everyone who invested also falls into at least one of these categories:
- Actual RJMetrics Users: Almost 100% of this round came from firms or individuals who have used our product extensively. From Jason Goldberg (Fab.com) to Ben Lerer (Thrillist/Lerer Ventures) to Andy Dunn (Bonobos/Red Swan), these investors had done their product diligence without even knowing it. They will bring practical, in-depth insights to our product development strategy from day-one.
- Stellar West-Coast Investors: Silicon Valley is underrepresented in our customer base, and we see that as a huge market for us in the coming year. Funds like SV Angel and SoftTech VC give us a west-coast presence and bring credibility, strong networks, and valuable regional expertise to the table.
- SaaS and Lean Startup Experience: Raising money doesn’t mean we’re going to abandon our mantra of capital efficiency. Investors like Mark Wachen (Upstage Ventures) and Gabe Weinberg have personally built and exited technology companies without depending on outside investors to survive. Their expansion-stage experience will be invaluable as we take things to the next level.
Why We’re Excited
We Love Doing This. Every day, we learn something new, interact with the smartest group of people we’ve ever known, and build something that changes the way people think. There is much, much more that we can do and nothing makes us more excited than working to turn those dreams into reality. This funding is only going to make that happen faster.
We Love Philadelphia. As we detailed earlier this month, we are proud to call Philadelphia our home and we’re excited about the emerging start-up culture here. We are also proud to be bringing brand-name capital from both coasts into the Philadelphia technology community. These dollars will be spent on cultivating local talent, creating jobs, and growing our hometown brand on the global stage.
We’re Just Getting Started. Raising a round of capital is not the finish line-- it’s the starting pistol. We’ve been working on this company for three years now, but we continue to start each day hungrier than ever. So, don’t say “congrats.” Say “good luck.” And stay tuned.
Here at RJMetrics, 2011 was full of growth: we went from a few employees to a dozen, and more than tripled our customer base. We moved into our new Center City, Philadelphia office on January 1st, and promptly outgrew the space. We’ve hired developers, analysts and designers, and although we’re far from a big company, we aren’t a couple of guys in an attic anymore either. Over the course of the year, we evolved a handful of traits in response to problems caused by this growth. These are a handful of the lessons we’ve learned:
Email is not an organization tool
E-mail was always our default medium for getting feedback from our customers and product. Customer has a problem? E-mail our support team. Code ran into an unusual exception? Send an automated e-mail to warn the dev team.
This approach has a number of problems. E-mails don’t have an owner so much as a number of contributors - how do we know who is responsible for this support request? How can we see all of the pending requests? Automated e-mails inevitably lead to a filtering problem - once I receive three automated e-mails with the exact same information, I ignore them going forward.
We have been transitioning our e-mail based processes onto more suitable platforms. Zendesk allows us to assign requests to team members, see all outstanding requests at a glance, and maintain a knowledge base of answers to commonly asked questions. Instead of e-mailing the team when our code does something unexpected, we automatically generate high priority Fogbugz tickets for our development team to investigate further.
Anything we can do, our customers can do better
In the early versions of RJMetrics, customers were unable to edit anything beyond the level of charts and dashboards - modifications to the actual data set had to be performed by an RJMetrics analyst. The stated reason for this was that the tools we had to perform these changes were powerful and confusing, a mix that we were afraid might lead customers to make irreversible changes accidentally.
But, as our customer base has matured, we have felt more demand for data warehouse management tools, and as our product development team has matured, we have recognized that the above reasons are really just lame excuses. We can build tools that aren’t confusing, and we can build in protections for reversing accidental changes - and that’s what we’re going to do. In 2011 we began by exposing a Trend editor, restriction set editor, and connection manager into the settings page, and we are planning to continue to continue to build out this tool set in 2012.
Bug tracking != Product Management
Just about a year ago we started using 2 week long sprints to organize development tasks and track progress. As our team and backlog grew, though, there was also a growing disconnect between small implementation tasks and our higher level goals for the product and company. Too often we would prioritize one or two of the most urgent tasks across dozens of projects into the sprint, and then feel disappointed when we didn’t have much tangible forward progress to show at the end.
To address this we created a road map of prioritized projects and categorized every task by project. Now we plan sprints by looking at the highest priority projects, and allocating enough time for each project to hit its predetermined milestones.
We are our own best critics
The most important change we made in 2011 was to introduce processes for giving ourselves feedback. Our code review system now ensures that poor programming practices are squeezed out, and best practices dispersed through our development team.
More generally, every month we conduct mini-review sessions to encourage our team to step back from their day-to-day work and discuss what they think is and isn’t working at RJMetrics on a strategic level. In fact, most of the changes listed above came out of these review sessions. This feedback mechanism will help us evolve and face the new problems that will arise as the team and company continue to grow.
These are a few of the most valuable lessons we learned this year - what did your company learn in 2011? Happy holidays from the RJMetrics team.
Posted by
Jake Stein on Wed, Dec 21, 2011 @ 01:34 PM
Since my first internship in college, I have made an annual donation of $5,000 or 10% of my post tax income, whichever is greater, to a charity called the
Smile Train. The Smile Train performs corrective surgery on children born with cleft lips and cleft palates in developing countries. The surgery costs about $250 per person, and it makes a huge difference in the life of the people born with clefts. Uncorrected clefts result in lifelong problems eating, speaking, and breathing.
Smile Train had
some drama this year when they announced (and later reverted) plans to merge with another cleft charity. I don’t know if this impacted Smile Train’s effectiveness, but the shakeup motivated me to re-evaluate my choice of charity, something I had been planning to do for a while.
I wanted to base my decision on charity
metrics. Using my fixed charitable budget, I want to get the highest possible return in terms of lives saved or drastically improved. I don’t give weight to the gender, ethnicity, or location of the recipients. I’m also not interested in spreading it around. I want to pick the best charity and give 100% of my donation to it. (For an explanation of the rationale behind donating to a single charity,
read this article by the economist Stephen Landsburg.)
One metric that I knew not to use, or at least not exclusively, is ‘efficiency’. This is the percentage of budget that goes to program expenses, and it's used by a number of charity rankings like
Forbes' annual charity list. While the efficiency ratio is useful for ruling out charities where much of the budget goes into management's pockets, it does not account for the big differences in return-on-donation between different charities.
Like any self respecting nerd, I tried to come up with a formula and make a spreadsheet. I wanted to compare charities on an apples to apples basis in order to figure out where my dollars would have the biggest impact. My initial attempt was based on the following variables:
U = cost per unit (For the Smile Train, this is a surgery on one person)
L = number of required lifetime units (An operation might be done once, medication may be require many doses over a lifetime)
E = efficiency (Percentage of the budget goes to the actual cause as opposed to overhead)
S = Odds of improving and/or dramatically improving a life (Vaccines are given to some people that would not have caught the disease anyway)
Cost per person helped = S * U * L / E
For the Smile Train, This worked out to be 1 * $250 * 1 / .822 = $304.
I ran into two problems with this formula. First, I could not find a scalable way to get inputs for large numbers of charities. The second issue is that a crucial variable was missing from my formula. What’s the difference in impact between preventing blindness versus increasing school attendance versus fixing a cleft palate? My formula treated all afflictions as equally detrimental. While that didn’t seem right to me, I wasn’t sure how to capture the differences.
While I was researching the inputs to my formula, I found two fantastic resources for optimizing charitable giving, both of whom had done much of the leg work already.
Giving What We Can (GWWC) was started by a philosophy professor at the University of Oxford named Toby Ord . GWWC makes charitable recommendations by trying to optimize a metric called
disability adjusted life years, or DALY. This is a much better version of what I was trying to get at with my spreadsheet. DALY is an increasingly popular metric to compare the burdens of disease and various treatments. It’s used by the
World Health Organization, World Bank, and others. GWWC also provides a
wealth calculator to give you a sense of where you stand in terms of wealth compared to most of the rest of the world. If you’re reading this, I bet you are richer than you think.
GWWC has determined the best charitable bangs for your buck are from curable and preventable diseases in the third world. Their three recommended charities are:
The other resource I found is
GiveWell.org. GiveWell was started by two former financial analysts named Holden Karnofsky and Elie Hassenfeld. GiveWell also uses DALY, but takes a somewhat more nuanced view to the charity optimization problem.
GiveWell uses DALY to filter down to the set of best philanthropic investments, but considers the error bars on DALY estimates to be too high to make a precise ordering. Once the universe of potential charities has been winnowed down to the highest-return organizations, GiveWell does proprietary research in order to attempt to forecast changes in the effectiveness of incremental donations. They interview management teams and find out about the projects they will fund if their budget increases. GiveWell reevaluates its top charities each year, requests updated information and interviews from the respective organizations, and updates its recommended charities list.
While there are differences in the methodologies of these two organizations, they come to similar conclusions. GiveWell’s two recommended charities both show up in GWWC’s list. In fact, GWWC recently updated their site based on research that GiveWell conducted. GiveWell’s top two recommendations are:
- Against Malaria Foundation
- Shistosomiasis Control Initiative
I decided to make my annual donation to the Against Malaria Foundation. GiveWell’s methodology of attempting to forecast the biggest DALY payoff for incremental dollars invested is, in my opinion, the optimal way to choose a charity. Shortly before posting this article, I funded my donation to AMF. I’m extremely happy to report that I will be responsible for the distribution of over 1,200 mosquito nets.
I encourage you to apply metrics to your charitable donations as you would for a financial investment. Or, take my word for it and
donate to the AMF now.
2011 has been an outstanding year for RJMetrics. We’ve tripled our headcount, creating eight new high-tech jobs in Philadelphia and filling our Center City office to capacity. We’re proud to have done this profitably and without the use of any outside capital.Today, we signed a new lease that will significantly expand our office space in The Philadelphia Building at 13th and Walnut. This was not a decision that Jake or I took lightly. 2011 brought with it a number of strategic opportunities, including offers that would have involved moving our company to New York or Silicon Valley. We turned down those offers and we’re doubling-down on Philadelphia. Not because it is the path of least resistance, but because it is the right path. We believe that Philadelphia is the best possible home for our start-up. Here are five reasons why.
1. Philadelphia is a Lean Startup’s Paradise
We’re huge fans of Eric Ries’s "Lean Startup" approach to building products and companies. If you’re not familiar with this entrepreneurial philosophy, it involves moving as quickly as possible to collect actionable information, acting on that information, and then iterating. It’s not about being cheap-- it’s about being fast. From day one, we were able to acquire actionable information faster in Philadelphia than we could have elsewhere. How is this possible? It’s simple math: time is money and it costs less to do business here. Rent is lower. Beer is cheaper. I live comfortably on a salary that would have me sleeping in a broom closet in New York. With a fixed budget, we’ve been able to conduct more tests, iterate more frequently, and get smarter faster -- all without prematurely raising capital.This effect is amplified when you remember that we sell a hosted product in a world that gets flatter by the day. RJMetrics has customers on five continents, but our physical location has no bearing on what we charge them for our product and what it costs to deliver. This leads to higher margins and more money invested back into growth.On a budget that would be razor-thin in many other cities, we can provide our employees with a world-class working environment, top-notch benefits, and some of the most compelling personal development opportunities in the region. This is helping us create Philly software jobs that tap into our city's tremendous pool of local talent.
2. Philadelphia is Rich with Talent
I’ve heard peers complain about a talent shortage in Philly. I agree that one exists, but a shortage of technology talent is not a Philadelphia problem. It’s a national one. We know startups in every corner of this country and the complaint is the same everywhere: demand outweighs supply, prices climb, and cash-rich companies like Google monopolize the top talent. Yes, there are a higher number of talented developers in New York and Silicon Valley than in Philadelphia. But it’s important to remember that competition for those developers is proportional—big companies will open their doors anywhere they can find clusters of talented people. For example, I wasn’t surprised to see Facebook's recent decision to open a development office in New York (although I said a prayer for my friends who are running startups there and already competing with each other, the banks, and Google for new hires).There are plenty of excellent developers in Philadelphia, but it is admittedly a smaller pond than other major tech cities. Developers do not gravitate here, but there are a lot of them here for personal or circumstantial reasons. We have found that population to be rich with ambitious, talented people.Our proximity to leading universities like UPenn, Princeton, Drexel, and Villanova also allow local companies direct access to top-notch engineering grads. Many of these talented engineers are "in play" every year. As we’ve grown, we’ve seen an increasing level of success with university recruiting.There are also a number of large technology companies in the area (from Comcast to Oracle to SAP) who have sizable technology teams. Professional recruiters are proactive about syphoning candidates out of these larger companies, and many startups like us have had success tapping into this pool of talent directly as well. Overall, I would say that the caliber of talent in the Philadelphia area is strong and the landscape of candidates is proportionally lower-volume and lower-competition than in other cities. In other words, the inputs look a little different but the absolute yield is comparatively attractive. We are building a world-class technology team here and see plenty of runway ahead of us.
3. Philadelphia’s Startup and Technology Communities are Thriving
When we moved to Philly in 2008 to start RJMetrics, the startup community was fledgling. An organization called
Philly Startup Leaders was picking up steam and we’d often bump into talented freelancers who spent their days at a coworking space called
Indy Hall.
In just a few short years, these groups have flourished into nationally-recognized organizations with hundreds of members. We’ve seen events like Philly Tech Week and the Philly Tech MeetUp surge into a successful existence. We've also recently gained exposure to some amazing mentoring organizations that exist for later-stage companies. In my opinion, the diversity of the organizations in the Philly tech community is a great thing. They allow a widespread population of technologists to each find a voice that works for them. For the most part, these groups peacefully co-exist and share the common goal of making Philly a more awesome place for tech and business.As these organizations continue to evolve and collectively impact the local economy, I think the pieces are moving into place for a coalition of tech players to exercise greater influence on local politics and work together on initiatives to raise outside awareness of the broader Philadelphia tech community. Today's appointment of Bob Moul to the presidency of Philly Startup Leaders is evidence of this progress. Stay tuned.
4. Philadelphia Companies Have Capital and Strategic Options
One of the many lessons Jake and I took away from our previous jobs in VC is this: there are awesome companies everywhere and, if your company kicks ass, it doesn’t matter where it’s located. You will have no problem raising money and you will have no shortage of options when the timing is right for an exit.
As we’ve grown, this truth has become increasingly clear. Any venture firm that invests in New York or Boston companies will gladly invest in a Philadelphia company. These include funds on both coasts.And doing business from Philly is easy. We have a major US Airways hub that provides easy access to the west coast and Europe. We also have extremely close proximity to New York, Washington DC, and the rest of the the eastern seaboard.Recently, Philly has seen big exits like Invite Media (to Google), Boomi (to Dell), and MyYearbook (to Quepasa). We have also seen major investments made into companies like Monetate (from First Round & Openview) and DuckDuckGo (from Union Square). Don’t be surprised to see more announcements like these soon.
5. We Love It Here
Philadelphia is an awesome city. It’s rich with history, has one of the best restaurant scenes in the country, and is full of great people. The arts scene is incredible. We have great sports teams and even better cheesesteaks.
We are proud to call it our home. Stop by anytime.
We are excited to announce that clients can now use our API to access the data from any of their RJMetrics charts. This will allow users to more easily integrate RJMetrics data into spreadsheets and other third-party tools. Data can currently be downloaded in CSV, JSON, and Geckoboard formats.
Many clients are already taking advantage of this new feature by displaying their charts in Geckoboard, a live status board that can be used to display important company statistics. Geckoboard offers a Custom Chart Widget which displays JSON-encoded charts. You can incorporate your RJMetrics charts into your Geckoboard dashboard in two ways:
- Export the data in Geckoboard format, which outputs the chart data in a specialized JSON format that can be directly loaded into Geckoboard using their highcharts widget.
- Export the data in JSON format. This outputs the chart's data as a JSON encoded list that you can load into a custom script that formats a Geckoboard chart to your exact specifications.
The result is an RJMetrics-powered chart like the one below available directly on your company's Geckoboard!
To learn more about how to use the new chart export API and how to integrate with Geckoboard, check out the
RJMetrics API reference.