Down, But Not Out

Ep. 1 How Google Dealt Visitor Queue a Death Blow

July 21, 2021 Nick Hollinger Season 1 Episode 1
Down, But Not Out
Ep. 1 How Google Dealt Visitor Queue a Death Blow
Show Notes Transcript

In our first episode, your host, Nick Hollinger, shares how his company, Visitor Queue, almost went under due to changes Google made to their APIs. This story was the inspiration behind the Down, But Not Out show and we thought it was the perfect story to start the series. Listen today to hear how Nick and his team were able to adapt and learn from their near death experience.

Hello everybody and welcome to the first-ever episode of Down, But Not Out, the show where founders tell their real-life stories of entrepreneurial resiliency. I started doing this show after my one company Visitor Queue was delta death blow by google that almost led to us losing all of our revenue and all of our clients and we would have gone out of business if it wasn't for our team's resiliency determination and hard work you see I love a good underdog so these stories resonate more with me than the one billion dollar exit or the story of how your company IPO'd, these are stories of grit meant to inspire you the listener to keep fighting as entrepreneurs and business owners we will have our backs against the wall multiple times throughout our journey so let hearing about others tough times be your fight song be your hoorah and that thing that gets you out of bed on those tough days this episode is brought to you by my company visitor queue identify the anonymous companies visiting your website so your sales team can follow up with them visit visitorq.com today to start your 14-day free trial so without further ado let's get started on episode one Now, I thought it'd be good for episode one to do my own journey, do my own near-death experience, my own time that I was Down, But Not Out and who am I well my name is Nick Hollinger I live in London, Ontario Canada and I started a company by the name of Visitor Queue that does IP address reversal to identify the companies that are visiting other companies websites and we sell this to a few thousand companies across the globe so that their sales team can follow up with those leads and hopefully close new business we've worked with some great companies some larger names like Microsoft and Jones Lang Lasalle and then we work with some smaller companies as well. We started in it was probably about June of 2017 that we finally registered the name and started building out our software and at that time it was just me and my co-founder Taras Zubyak that were working on the company and Taras was building out our MVP as I got other things ready for our launch and we were able to launch a beta of we actually launched an alpha first with a few hundred or a few hundred testers in about September of 2017 and then in December of 2017 we launched our full beta and opened it up to a few thousand more testers and after that and after some feedback after they broke the system they were eventually ready to launch into our full paid product in April of 2018 and this is being recorded in what is it March now of 2021 and so about three years after the launch of our full product and our Down, But Not Out story happened in February of 2020 and I remember that day like no other, very cliche I remember it like no other but it's absolutely true. February 5th, 2020 I just came off a dry January so I was feeling like a million dollars getting more sleep than ever feeling great and I get into the office, that morning I had shared an article on LinkedIn or a post on LinkedIn that talked about how to become a Forbes 30 under 30. I looked at all the stats from previous years and came up with this is how people become Forbes 30 under 30 because it was of interest to me that's something that I wanted to do it's something I still want to do and something I'm working towards and I just thought it was ironic that right when I shared that this is what my day turned into. So I get into the office, I'm sitting there and I'm working away on something, don't remember exactly what it was but our one Marketing Manager, who is also in charge of customer support, starts sending me messages or maybe turned around maybe he's working from home that day I forget, but he says hey our clients are complaining they haven't seen data in their dashboard for for 12 hours or something like that and because we were built on top of Google Analytics to provide our service we were used to delays in their APIs, and that actually might be a good time to pause and get to a bit of filler to explain where that issue came from, so as I said we do IP address reversional to identify companies visiting other company's websites, how we did that IP address reversal previously was we had to use Google Analytics so we would integrate directly with everybody's Google Analytics account to provide our service and we did that via their APIs because it was much easier to integrate with Google to provide the service than trying to build out our own Google Analytics alternative, so yeah that's that's the little bit of back filler there so I'm sitting there at my desk, they're saying hey we're not getting support requests or we're getting a bunch of support requests about not getting data for for 12 hours or something like that we were used to a delay of five to six hours at most from the Google Analytics APIs, it was out of our hands, we couldn't do anything but that was a normal thing that we were used to, 12 hours was way too long so I start looking around and I messaged our developer right away and I say, "hey like are you seeing anything on the back end where is the issue here? Can you tell me about what's going on?" as I kept looking on the front end to see what I could see to multitask to tackle this in multiple different approaches and he went looking through and he said we're still getting data back from Google Analytics so I don't see where the issue and he started looking at other other areas where there might be issues and I start digging around into actual Google Analytics on the front end to see okay what does the data show on Google Analytics in our system, and I'm in there and I'm looking at the reports that we usually get and I notice that there's two fields that Google previously supplied to its users and were previously available in the API because we've got about 10 different fields from Google Analytics what you did on a website the pages you viewed and then a network domain and a service provider and so on but I'm looking at this network domain and service provider report and it's coming up with what do they say they say not set that's Google's classic like we're not going to tell you what this is they always do this hey you want to know what keyword someone used to get to your website not set google's not telling you that they they put not set when they're trying to hide that data from you and all the power to them that's that's their that's their platform they own that space let them do it that's that's on them but it was never not set before I've never seen the service provider network domain report the two fields we absolutely needed to provide our service come back as not set so I became alarmed and I start looking around through other reports not noticing any changes there so google's not broken I was hoping google analytics was just broken not the case so I looked start looking on forums I started Googling like hey network domain service provider missing yada yada come across a forum it was somewhere in a Google Analytics forum, so Google owns the forum, where people could ask questions like a community little section for everybody and people are also mentioning hey I haven't seen network domain or service provider come back for a number of hours it's all coming back as not set and I'm seeing post after post of people confirming this and then I get a little alarmed, so I message our developer and I say, "hey are you not getting these fields back? You're getting everything else other than these fields?" and he goes yeah and I go well that's going to be a problem, that's not good and I was holding on to I guess some dumb optimism that they had made a mistake because I'm a big fan of Google, still to this day can't hold it against them, they usually tell you when they're going to sunset something when they're turning down, they're turning off a product, they're making a change that's going to break something, they're usually pretty good at telling their clients. I've seen it multiple times hey we're sunsetting this we're getting rid of Google Plus, no one really cared, but they told everybody they're getting rid of Google Plus so I held on to some optimism now they would have told us if they were getting rid of those two fields and I call our rep at Google and I say, "hey like we haven't seen this field what's going on?" and obviously they're not going to know right away, Google's a very large company so they're not going to be able to go yep we removed that this morning yadda yadda so they have to ask a few friends of friends of co-workers and and work through the Google chain of command to see what's going on here and the next thing i know we get we get an email back from Google and say yeah those are removed they're not coming back and I remember the weakness in my in my knees the feeling in my stomach and like the pure anxiety, reliving this right now is a bit of PTSD and this hurts because we were at a few hundred thousand dollars in recurring revenue, we had I think five or six employees at this point we were growing 400 percent year over year, we were on a really good trajectory to do seven figures in 2020 and really get up there in reoccurring revenue and now we were back at zero without those two fields in the Google Analytics API we were screwed we were completely out of business we didn't have a service to provide to our 250 clients that some of them had prepaid 30 of them had paid for a year so that means we owed them money in order to shut down we would have to refund them or go bankrupt so I'm sitting there thinking and just going through this going through all the scenarios in my head and luckily for the for the first few days at least I had this this killer drive, it just clicked in, it was like a fight or flight and I stepped up, I stepped up and I was like this is time to fight I've been in, I've been in flight mode don't get me wrong but this I knew what was coming out was fight so I rationalized I sat there and I said okay what are our options and I came up with four different options, we either shut down and walk away, we try to sell our client base to a competitor, or someone else that wants to buy our client base for probably nothing because they know we're screwed they're not going to give us a good multiple on revenue that doesn't even exist at that point and actually one of the competitors did message us in the middle of this and asked to to purchase that client base but we never went through with that. The third option was to okay we have a bit of money in the bank we have more debt than we do money let's try to pivot and try to build something else but that that would be incredibly hard to do as any entrepreneur listening to this can can attest to getting product market fit to begin with is next to impossible pivot pivoting in the middle of a crisis to try to get product market fit next impossible and the fourth one was okay we have to rebuild what google analytics was providing us and to to give a hierarchy it pretty much starts with Google Analytics telling us these are the service provider the network domain of the companies visiting this website the pages they viewed the amount of time they spent the source and medium and so on and then we take all of that we clean all that data so we still have a bunch of IP that just is below Google Analytics we clean that we process it we filter it and then we display it in our dashboard or to APIs or marketing automation platform integrations to show to our clients so we still have a lot of IP and great stuff that we've built it's just Google was that first plug that we needed and it was it was removed, so the fourth option was to hey if we rebuilt what Google Analytics was providing us then we should be we should be in a good spot to make a recovery from this so we weren't absolutely screwed we had options, so I take these options to our co-founder Taras Zubyak who was our CTO at that point, and more of a fractional CTO than anything we had our own team of developers that were working on the product, and he had his other businesses so he would he was focusing on the other businesses and that was always the agreement so I took I took it to him as the second largest shareholder and the the technical co-founder and I said "hey we have these four options," I presented them the options and I was in fight mode at that point so I knew we were going with number four and I was going to convince him to go with number four but Taras is a fighter as well and he said we there's only really one option there so we're going for it and we we agreed upon that would be the approach so the next thing I did was I brought our team together and I said guys this is it this is the this is the end game we have no other option other than to rebuild what Google Analytics was providing us as quick as possible otherwise we are completely out of business to put it into some context we had about I think 250 clients at this point none of them were getting data none of them were getting data a lot of them check their their their dashboard every day to see what what new clients visited their website what new companies visit their website they get emails every day they show that they're not getting any of that so they know something's wrong every minute every hour that goes by with no data showing up they know something's wrong so I said guys we have to we have to rebuild and if we don't then we're out of business and I don't remember exactly what I said but I knew I know I made it as sound as dire as possible because that's what it was I made sure they understood that this was the ninth inning step up to the plate or were screwed and I'm gonna be honest I've never been more proud as a leader than I was that day because I was able to rally our troops to step up and fight with me and after getting them all rallied up and telling them this is what we're doing we laid out our tasks of how to rebuild this component and it might sound easy to some but some technology enthusiasts out there understand building a javascript tracking script and tying it into an existing system isn't easy and to put it into con and put into perspective for everybody this should have taken us three to four months to build this replacement for Google and we had negative 24 hours at this point 16 hours at this point negative because we had already gone that long without showing data to these to these clients that we had to what we've built over the past two and a half years and yeah we were able to rally our troops get them get them ready to go and lay out an action plan of these are the tasks that need to be completed in order for us to to build this and to replace Google and and to survive and to fast forward a bit we started we started building it out the next day we had developers and our lead developer working 20 hours a day to to try to push out a replacement for google in a in a time where we didn't we didn't lose all of our clients and speaking of our clients we originally once we saw there's a and this is common whenever we see there's a delay in data that's abnormal we would say we put a notice to them on the dashboard that says hey we're experiencing an issue with data, time coming back latency and that was normal so they they expected that but this long wasn't so we put it we put that notice up until we figured out what happened once we figured out what happened we sent out an email to all of our clients saying hey google made a change in the apis that prevent us from using them moving forward we're building out a replacement you'll have it hopefully in the next few days we apologize you'll be refunded for any time missed plus some and when we sent that email out I think it was either the next day or the end of the first day once we actually figured out exactly what's happening in our strategy we locked the the churn notifications for anyone in SaaS, the cancellations I'll just say that we're just pinging just going and going and going and I don't blame clients I don't blame our clients for leaving us when that happened that makes sense to me but all that was going through my head while that was happening was okay we're still not back at zero this is what we've worked on for the past two and a half years we went from 250 clients we might go down to 10 but at least we're not at zero we're still above zero which is good fast forward to to about three days after this had happened we're building it out we're still losing clients like crazy we're we're refunding them not charging them for for what for this time that they don't have data and the weekend approaches and I know the weekend's a crucial time because clients aren't online on the weekend, we're strictly B2B, so clients aren't going to be online and checking if they're missing data and canceling on us because they are so I know the weekend is when we can take make up time for the for the loss for the lost data while people are enjoying their weekend we can be building out our script right so but our our team has weekends off just like every other B2B company they have weekends off they have families they've got stuff they need to do so I sat down with our team and our lead developer predominantly and I said like we don't have an option here we need these two days to make up for it in launch monday or tuesday otherwise we're we're not going to be a good spot with our clients we're going to keep losing we're going to keep burning clients and I think we lost about five percent of our clients at this point and again I made it sound dire I leveled with them I told them like this is it we're going out of business unless we can we can pull this off and they agreed to work throughout the weekend and our lead developer mostly was the one crushing it out the the 20 hours a day working on the weekends there everyone jumped in though everyone jumped in and helped I won't discount that and over the weekend we were able to put together something that's mostly usable we pretty much went from having an established product something that was good definitely passed the MVP stage of your first launch to pretty much having an MVP again we had to replace a major source of our data the initial source of our data so when we when we relaunched on that monday after the weekend we were pretty much back to an MVP and we cut some stuff that we didn't need we didn't have source and medium and we didn't have some other fields we didn't have the ability to control your time zone, which we lost clients over believe it or not, we were missing a lot of stuff because we had broken a major piece of our system and put in this hack together solution we launched it monday and started testing it out and then put out a notice to our clients that says hey you can now see data again but you have to add you're tracking this tracking script, the Visitor Queue tracking script, the new tracking script to your website to the header of your website because previously we just used Google Analytics, 98 of websites have the Google Analytics tracking script on their website so that prevented us from having to add a tracking script to their website now that Google Analytics is gone we have a tracking script that needs to be added to every one of our clients websites and when I sent out that email ping ping ping ping ping all the churn notifications again of people upset that they had this tracking script or just noticing that they weren't getting data they didn't see my previous emails and so on but we were able to keep about 80 to 85 percent of our clients which again it hurt, it really did but the thing I kept telling myself the entire time was eighty percent is better than zero anything at that point was better than zero because you were pretty much at zero don't forget that you're pretty much at zero and even though it only took five days those were five or six days those were probably the most sleepless stressed out times of my recent of the my recent history they were they were brutal I think the first two days, two or three days I had that fight and I was ready to go but once my team picked up and didn't need me to have that that fight on my face and showing at all times to them i was drained I kind of realized what had just happened I almost lost two and a half years worth of work I almost lost a few hundred thousand, two hundred thousand dollars and 250 clients everything I'd worked on everything I dreamed of was was gone and once that clicked in a bit of depression the the feeling of just pure anxiety at all times not sleeping just kicked in and I was able to work still which was good I could work through it but I couldn't show optimism it was very hard for me to show optimism I could fake a smile I could push through but for those few days after that initial piece of strength it was very tough to show optimism we as I said we launched our script we were good to go but it was an MVP script so we kept losing clients we weren't adding as many clients in after we were missing a bunch of pieces and after that five or six-day ordeal we progressively got better but it probably took us 10 months to fully catch up and we're just now catching up to where we were previously so this has been a 10-month ordeal a 10-month down but not out story give or take where we were growing 400 year-over-year and now last year we grew 30 percent I will say that this was February of 2020 and everyone knows what happened in march of 2020. so it was almost a one-two punch Google hit us and then COVID-19 hit us but that's business that's life really and it's probably not going to be the last time that happens to us so we lost about 20 in the ordeal we're up about 30 in 2020 and in the first three months of where we are now in 2021 we're up already 30 in 2021 so we're now getting back to that growth trajectory we're at about five employees before we're at about nine now with full-time equivalents and so on, it's always good to hear those numbers always good to hear that fighting back side of stuff I just want to touch on a few points that I'm going to be asking my guests on this show the one was considering quitting and I don't remember considering quitting I remember it sucking I do remember it sucking and that pain and like I said this is all reliving it just hurts to bring up at times it's a great I think it's a great heroic story to share and something that might inspire somebody that's going through something similar because these are the stories I love to hear but I don't remember considering quitting I remember it sucking but I don't remember considering quitting i've never been a quitter I will say that I don't remember anything that i've really quit I always stick it through I always fight I've always been a fighter I've always been able to step up there and not a great fighter not physically but if you're if you're pushing, if I'm getting pushed mentally and physically i'm gonna fight through it that's just kind of who I am and something that I've always had with me and it yeah that's that's probably what got me through it was the fact that I'm stubborn as good hell you could ask my you could ask my fiancee and my friends it could be the dumbest challenge it could be the dumbest competition but I still want to win I'm still stubborn and stubborn gets a bad rep it really does like a lot of the best accomplishments was because someone was too stubborn to give up there's times you can be too stubborn if you're not admitting you're wrong okay fine that's bad take it too far yeah absolutely if they can prove you're wrong give it up you're stubborn in a bad way but if you're so stubborn that you're not giving up even though the odds are completely against you like in our case like in many of the other stories you're going to hear from this podcast I already have some great people lined up that i'm looking forward to talking to then then stubborn's a good thing you got to consider that stubborn's a good thing and then lastly what I learned from this experience i just want to say and I will mention this before I get to this other part about what I learned from this experience that I shouldn't have built on top of someone else's platform everyone's going to say everyone any of the good entrepreneurs out there will go well you built on someone else's platform they can make changes that's your fault you should have had a replacement ready and you're right we should have had a replacement ready but we didn't we were chasing growth we were hitting 300 400 year over year, you're in that position you're gonna be like okay I'm gonna pause my investment into our marketing and feature set that's to keep driving growth to build this contingency plan that's going to take up two to three months worth of my developers time I don't know if you're going to do it if you are good for you but I wasn't willing to do that I was willing to keep growing and running that risk I always knew this could happen this wasn't a complete shock that it could happen it's the fact that it did happen and Google didn't tell us and it hurt probably more than than I was anticipating as well, so yeah if you're saying out there you should should have known it was I knew it was gonna happen yeah it could absolutely happen to anybody building on top of someone else's platform it could happen so what I learned from this experience is if you are building on top of someone else's platform have a backup plan or be okay with the fact that they could drop you any day either either build that backup plan that we probably should have had ready but we didn't or be okay with the fact that they could drop you one day mentally prepare yourself for that worst case scenario because it will likely happen if you're built on top of someone else's platform and reliant on their APIs or for them to drive your your client growth then that could change in an instant and your business could be thrown through a loop and you could be on down but not out okay I'm gonna wrap up there thanks for listening and I look forward to sharing more down but not out stories with you guys in the future cheers guys hey guys thanks for watching episode one of down but not out we just finished editing and it turns out that Sony cameras turn off after about 20 minutes or so of video so I hope you liked the animated version of me hey Siri remind me to buy a Nikon today at 6 pm ok added