5 Things to look for in a Computer Vision startup job

So you’re looking to join an ML startup in computer vision. At V7, we partner with hundreds of the most ambitious AI and ML startups bringing sight to machines. Here are the common traits that the highest-achieving ones share:

1. Do they have data?

This should be your starting point. An ML startup without data for you to use will make your job harder and no fun. During a job interview, make sure you understand the data you’ll be working with very well. This doesn’t mean revealing its content, but you should know in general abstraction about:

  • Where it’s coming from (is it exclusive?)
  • What it’s captured by (is the device relevant?)
  • What visual characteristics is the startup looking for (can it be labelled?)

Don’t be rude – AI companies are protective of their data, so rather than asking “Where is your data coming from” consider asking “are the sources of the data your users? What kind of profile do they have?” Exclusivity of data is a huge moat for AI startups, so if the data being used comes from an open dataset, consider that a red flag.

Sometimes the device capturing the data may be the product – a robot, camera system, or appliance. In this case, make sure that you’re able to work with that hardware configuration before jumping in. Consider asking whether the hardware is fixed in place, or if they’d like you to experiment with it.

Finally, whatever they wish to identify must be supervised in some way, else you’ll never know if your model is working. Make sure the data can be labelled within reason. If the objects can be enclosed in a polygon, that’s great. If temporal understanding is needed, that’s ok, but may not be elegantly solvable today. If the data comes from multiple cameras across large spans of time, with occlusions, appearance changes, or other obstacles, consider that this may be a research problem only solvable with a human-in-the-loop today.

2. In what state will you find the data when starting

Hopefully not like this

There may be none, there may be raw images, there may be poorly placed labels or a combination of the three. Rarely is computer vision talent hired once a dataset is ready to go. In fact, this probably never happens, you will have to give your input on how its labelled, organized, and optimized once your product launched and racks up failure cases.

V7 was built to help you with this stage. You can automate labelling, organize and manage huge datasets, version control your work, and load it straight into Pytorch or Tensorflow.

If your data is raw: The good news is that labelling data is very quick now. You can use something like Auto Annotate to apply labels rapidly on objects of interest, or load pre-existing labels from a model output.

If your data is labelled: This isn’t always good news. Ask yourself who labelled this, and for whom? Was someone dropped from the project who was supposed to use the data?

If there is no data: You might be twirling your thumbs for a while, or working on proxy datasets while it is collected. Encourage the team to capture some as quickly as possible for validation purposes.

3. Figure out if you’re in research or engineering

Source: Experfy via SalaryNinja (https://www.experfy.com/blog/machine-learning-engineer-vs-data-scientist-is-data-science-over/)

There are no “research engineers”. There are engineers asked to implement papers, or researchers asked to develop something for the user – neither of these tend to be happy folks.

Most talent in computer vision likes the idea of research because it sounds more noble, and less prone to deadlines. Commercial research also has a dark side – it is a more volatile job: your research topic moves quickly, and you might not be needed next year. If the company has a team of researchers that cannot be counted on 1 hand, make sure that they have a killer revenue stream, or flush with capital. Researchers are expensive and distant from directly generating revenue; when ? hits the fan, they are often the first to go.

Engineers are more loved when times are tough, and in most industries they have more career opportunities. Startups hiring “computer vision engineers” or “deep learning engineers” at the very most expect you to implement the cutting edge (whatever that means for them), but not iterate on it.

You know already if you are one or the other – make sure you apply to a position that suits that profile. If the team you are joining is less than 14, it is inevitable that you will have to do a bit of the other.

4. Is the problem in a space you care about?

If the tech is applied, make sure the area is something you find interesting, because it will define how you will introduce yourself at parties. If it’s fundamental research, ensure that the vision that it leads you is one that excites you.

Be wary of startups that sell you the idea of an idyllic product that changes the world effortlessly – you will have to work hard, on hard problems, against fierce competition of other motivated fellow ML engineers. Seek instead those that give you a glimpse of what a hard day on the job might look like.

Even if your problem appears very fundamental, it will become applied one day. Your mom and dad will recognize the application of your AI product more so than the tech behind it, and that will likely be the first use-case to hit the market or the press. If your groundbreaking hand tracker will primarily be used to make dance videos on apps, would that excite you? If not you are doing yourself and your employer a disfavor by entertaining that role.

5. Do they have a chance of becoming N1?

More importantly, do you have a chance of making that team the best in their field?

Your employer will likely be part of a niche problem space that they aim to dominate – something like identifying hand signs, unusual defects, triangulate a location via camera, signs of climate change, the wake of ships, picking ripe strawberries, car’s dents… there are hundreds of huge problems to solve that require sight. Vision AI is an exciting field where new use cases appear every day, and hundreds of startups emerge to resolve them. Does the team you are about to join have a plan to be the best, or the largest of a handful of players in a space? Have they implemented the right tools, engineering practices, and data flow to help you help them succeed? Are you going to help them make a dent in their market so big that people will notice?

You aren’t looking for clear “yes” answers here – nobody is proclaimed a winner at the starting line. However make sure that the questions above aren’t clear “no”s. Most important of all, don’t go to the team asking for answers – do your own research, and come to them with a hypothesis. For example, rather than asking “what are your thoughts on Competitor A?”, say “I’ve looked at Competitor A’s product and it looks like they’re missing out on using this approach, is the goal to use it as a differentiator and become better than them?” This will grant you some brownie points and a less defensive answer, and if you’re wrong, you’ll learn more about your interviewer by the way they correct you. Remember, you’re there to be helpful, take that opportunity to see what that feels like in return!

Good luck!