Thomas Hazel

From telecom to startups

Hi, I’m Thomas Hazel, the founder and CTO of ChaosSearch. I am a classically trained computer science nerd and have always loved to solve some of the hardest problems.

After graduating from college, I was in the Boston area and began my career in the hot telecom industry building monster boxes. From there, I started and sold a company in the virtualization space to Oracle that involved an open-source database I had developed. As a result, I came to realize I have always been innovating in database science throughout my career.

It was here that I jumped into creating my own startups, which connected with my natural entrepreneurial side. I saw opportunity in the database limitations. When I dove full time into database innovation, now over 15 years ago, 1970s computer science was still state of the art.

Back in 2012, I ended up taking off for about a year on a pseudo-life journey to crack the code on how to make data more accessible, more cost-effective, and easier on a larger scale.

A fascination with data

I don’t know why I’m so drawn to the science behind building databases, but it’s gotten to the point where everyone makes fun of me every time I do a startup. I love data, I’m fascinated by information theory and the fact that everything is information. It started when I was a kid and my friends were joining bulletin boards in the 80s where one could find video games and software—I thought it was pretty cool. I didn’t want to necessarily play these games, but understand the software that made them work. One thing led to another and when I had to decide on a career, I knew I liked hacking computers/software so I went with computer science. It was more accidental, whereas now computer science is so popular. At the time, It was not the coolest thing to be involved in, but it is where I found like-minded friends.

Starting with first principles and ending with ChaosSearch

If you know me, you know I always like to start with base principles and then dive deeper into finding solutions to problems. Knowing that data continues to get bigger and bigger, I wanted to try and make it super small. Spoiler alert: it wasn’t that simple.

I worked on a new data representation and compression algorithm to make any data source as small as possible as quickly as possible. With my background in information theory, this was fun for me. I was tinkering trying to make data small, I then had an awakening. I separated information into two core components: symbols and locality. When I did this, I achieved the theoretical minimum size.

When I looked at this unique separation, I realized symbols would support high performance search for log analytics. I looked at the locality, and saw I could do performant relational mathematics for business intelligence…my mind started simultaneously wandering and exploding. I knew that with my next startup, I was going to do something with this technology and put it on the shelf just like any other entrepreneur.

About five years ago, the timing was right. I had a strong team of engineers that I had been mentoring and I reached out to the VC community to explain my crazy idea: transforming cloud storage into a big data/multi-model analytic database. Lo and behold, they agreed it was crazy, but saw the potential. And with a several million dollar seed, we built out this technology with a new distributed fabric on cloud storage. Finally, about two to three years later we turned on the engine and it worked perfectly. This was the origin of ChaosSearch.

The ideation process

Everyone comes up with ideas differently. I like to look for a problem, see what has been done before, and then go the complete opposite direction. Invert the problem to come at it from a different perspective. You’d be amazed at the discoveries that you might find. It’s like an artist using their hand as a lens to alter the perspective—changing context lets one see things or problems in a different light or angle.

I took this ideation approach in one of my previous companies and ended up building a new tree data structure/algorithm for faster database indexing. However, it still had the principle of adding indexing to find things faster. And yet the by-product of over-indexing increases the database size and such structures reduce compression ratios. I realized that I needed to remove both the classic indexing and structure to solve the “next next” level scale. This was the beginning of a new idea I had and where I inverted the problem: I merged these two concepts into a compression-index hybrid.

Failure doesn’t exist

I honestly don’t see failure at the end of the day: I see it as an aspect of learning. If you’re writing algorithms and something doesn’t work, is that a failure? I say no: you just learned something that will make your logic better. Everything is a point of learning and growing.

This mindset is also something I see amongst other entrepreneurs and how they view limitations. They take something that is considered a weakness of theirs and work hard to make it a strength. For instance, here at ChaosSearch I was initially given several millions dollars to make an idea into reality. That’s a lot of money to spend on an unproven idea, but entrepreneurs have the confidence and mindset to jump right in and convince others to jump in with them. It is a lot of responsibility, but that’s what makes a great entrepreneur.

You don’t know until you turn the system on

When I was creating the index technology that makes up ChaosSearch, I always had the belief and faith it was going to work. When I did some smaller-scale tests, it was doing everything I thought it was going to, so I was feeling optimistic. But because we were building a distributed system, you can’t test it until you turn it on.

So I remember the day I first turned on the engine so clearly: it was a Saturday morning, and I had gotten the last piece to click in with the whole team in tow. I did my first query on pure cloud storage, not in cache, and it was fast. Really fast at scale. I was finally able to take a deep breath. My wife and I are based in Boston and always go on walks down the Charles River, so that fateful day we went on a walk and I turned to her and said, “This is going to be a game changing company.” Everything just came together after that...the rest is history.

Overcoming new challenges

The problem we face at ChaosSearch is that we are operating in brand new territory, so we hold our breath a little with every new step. There’s no way to Google a problem to figure it out and see how people have done it before because we are establishing new processes every day. Everything we do has a little bit of a “Eureka!” feel to it, like oh wow it worked! The difficulty comes from the complexity of big data scale and performance. Most of our customers work in PETABYTES worth of data. One petabyte is 1024 terabytes, and for a quick comparison, the Library of Congress is 10 terabytes of data. When every new step is a challenge, it almost feels like there aren’t any challenges.

Reliability and incident practices

A major driving principle at ChaosSearch was to never own a customer’s data at rest. We use cloud storage such as AWS S3, and can easily/quickly connect to their storage. And like any good database, we wanted storage to be the most secure, durable, and reliable storage. Cloud object storage was the perfect fit, we just had to figure out how to make it fast (the innovation). Architecturally, the distributed fabric that we have is expected to fail at all points in the system. This way, there is no single point of failure. We could kill every node and the system would still operate.

My advice to those just getting started with reliability practices is to make sure you keep up with whatever your plan is and to test everything, especially as you scale. Test failure, kill things, test change, test upgrades. All of these things seem so obvious, but we are all in a rush and the first things that typically fall off when we are focusing on bigger and brighter horizons are reliability and availability.

With my telecom background, I learned that you also have to be fault-tolerant. Failing has to be integrated into your plan, so that if or when it happens, it’s expected.

Looking to the future

The success of ChaosSearch has been great, and I am thankful for that. What surprised me is how good it felt to hit new milestones and continue to grow because we risked so much to get here. It’s been wonderful to see our work resonate with customers, and we are going to continue aiming for the sky in everything we do. I truly think we are going to change how people do business and manage data.

Being the head cheerleader

Everyone’s a little different in their style of leadership as a founder. I’m here to enjoy the journey and solve hard problems while also proving that I can make a strong business model out of it. I use my energy and enthusiasm to motivate people, and I want to work with people that I like hanging out with. I seek out like-minded people that want to join my journey and inspire them.

Of course, this needs to be balanced with the responsibility to investors and others that expect you to deliver on your promises, but it is possible to do it positively. For me, it is all about driving momentum. The moment you stop (or sleep) as a founder or as an entrepreneur, the company stops. You have to be the number one cheerleader for yourself and your company so that people believe in your cause. When things go wrong, everyone gets worried, and you have to be the one who says that it’s okay and we are going to work through this together until we find the solution. Always keep on moving forward.

My main advice for those looking to solve problems and be an entrepreneur is to find something that inspires you and do the research. Consume all the information on the subject that you can, play with it, and always be learning more about it. The more you understand, the better you’ll be at it. Experience is everything. Don’t get me wrong, school is important, but there is a lot to learn that can’t be found in formal education.

Advising work

I love meeting fellow founders or soon-to-be founders because many just have that “it” factor. I am always drawn to divergent thinkers, so I see advising as a way to hang with like-minded folks who also love pushing the envelope. It’s like a drug, and I love being able to provide insight from experience. I almost do it less because I’m looking to help, and more because I am so fascinated with their passion and love to be involved in any way I can.

Thomas recommends

When it comes to games, I grew up playing on an Atari-400 computer in the early 80s but then discovered quickly that I liked learning about how the games worked more than I liked playing them. I’ve also never been a huge board game guy, but I remember loving Chutes and Ladders when I was younger! Haven’t had much free time to dive into other new games otherwise, but that is always a classic.