Ruminations #11: Two experts on where to build in AI and Andrew Ng's advice at NeurIPS

Supercycles and how to survive them

Hi all,

Welcome to 2024! I spent my break with family playing Monopoly Deal, surfing and addictively making my way through the levels of this CPU-building game Turing Complete.

Before we kick off can I ask a favour? I’d really love to hear what kind of content would be helpful for you here. Do you want to hear from VCs investing in AI? From researchers in the space? From our team on how we're shaping our thesis? Something else? Your feedback would mean a lot to me! You can reply to this email.

In this edition:

• Notes from a great write-up on technology cycles and a paper on generalist models vs domain-specific

• Expert Kendra Vant shares what she’s looking for in AI startups

---

Content we’ve enjoyed

Capex & Opex supercycles — the dusk of SaaS and the dawn of AI-SaaS

Thank you to the researcher at Canva who sent me this great piece from a Partner at Lightspeed. It’s quite relevant to founders. Below are the excerpts I shared with the Square Peg team:

He centres much of the piece around these three stages:

1. Schumpeterian forces that create what are ‘extractive’ industries — industries that create the new as they destroy the old, strive to produce more with less, and unlock some net-new resource for the world.

2. Next comes the Ricardian forces that take this new resource and distribute it to new markets, make this resource accessible to all. Generally these forces deal in trade, expansion, and competitive dynamics.

3. Finally, the Malthusian forces create fear that there won’t be enough for everyone. That as resources expand, so shall the need for it, till there isn’t enough for everyone. And thus the cycle goes back to ‘extractive’ — extract something else, extract more, saturate, restart.

These were his three finishing thoughts:

1. My first AI project was back in the early 2000s. I’ve seen a few of these hypes and winters come and go. 2024 will be another winter but AI is a one-way street now. In 2005–10 you could be a social media sceptic and survive as a CMO. No more. In early 2010–2015 you could be a cloud sceptic and survive as a CTO. No more. A few years ago, you could be an AI sceptic and survive as a CEO. No more. You have to be all in.

2. If you are building in the ‘extractive’ part of AI, great — $$s will chase you more easily. But you are also likely to suffer from the great ‘pinch-point’ of the hour-glass that’s coming. It’s good to play the picks and shovels game in a gold rush but how many picks and shovels companies do you remember? This is the category that’ll likely see a lot of M&A in the coming years as big platform players will do tuck-ins. As a founder, your job would be to build fast into the available gap today, but then start to sprawl horizontally to build a platform quickly.

3. If you are building into the ‘distributive’ part of AI, I guess the obvious solve here is that you need to own some of the data, the models, or own parts of the training or inference stack that gives you an edge over others to begin with. In other words, you need to get closer to the ‘extractive’ part of the layer to get to faster value accretion. Another strategy is to just play the long, long game. Maybe raise less, take a few years to get to scale profitably, and once the lower layers in AI stabilize and we are clearly entering the distributive phase of AI, raise a larger round and scale up!

Can Generalist Foundation Models Outcompete Special-Purpose Tuning?

A researcher at Meta shared this paper with me over the weekend, knowing that I have an interest in what data-based competitive advantages will look like going forward.

The argument I hear from founders is that, with enough data, you can outcompete your competitors with more performant models. I think that’s less true than ever thanks to foundation models.

Here’s some of the abstract from the paper:

Generalist foundation models such as GPT-4 have displayed surprising capabilities in a wide variety of domains and tasks. Yet, there is a prevalent assumption that they cannot match specialist capabilities of fine-tuned models. For example, most explorations to date on medical competency benchmarks have leveraged domain-specific training, as exemplified by efforts on BioGPT and Med-PaLM.

…

Rather than using simple prompting to highlight the model's out-of-the-box capabilities, we perform a systematic exploration of prompt engineering. We find that prompting innovation can unlock deeper specialist capabilities and show that GPT-4 easily tops prior leading results for medical benchmarks.

This loosely aligns with what I heard from Andrew Ng while at NeurIPS. His advice was:

• Know that fine tuning is for changing the style and tone, whereas retrieval is for accuracy. Andrew suggests working hard with prompting and use fine-tuning only if necessary. Think of these models now as reasoning engines, not as memory machines.

• Additionally, don’t assume that short prompts are the clearest prompts for an LLM. Generally longer prompts are helpful as they offer more detail.

• Consider that AI doesn’t automate jobs, it automates tasks. Don’t focus on jobs to be done. Focus on tasks to be done. Use that to stitch together solutions, as opposed to try and have it solve everything. Don’t look at job descriptions to understand what tasks your customers need done - look at what they actually do each day and then rank those tasks by their relevance to solving with AI.

• Going forward Andrew is excited by customised models, in which models actually get to know you. He’s also excited about more complex agents in which there are deeper sequences for completing tasks.

---

Expert Q&A

This is a new section I’ve started where I’m sharing answers I’ve received from experts in the field on how they think about all things AI. This edition I asked a question of Kendra Vant who describes herself as a “forever curious lapsed physicist with an insatiable interest in real-world impacts and seven years spent building AI-powered products in SaaS companies with global reach.”

I love her response to this week's question. I asked:

If you were an investor, what types of businesses would be most interesting to you? What would be an indicator of an interesting business to you?

A lot of founder teams are looking to play in the Generative AI developer tools space. This is understandable given the enormous amount of attention focussed there and the potential to build a hyperscale business. However I don’t see it as an area that provides great opportunities for startups just getting going today. It’s too competitive and data gravity, funding and access to compute tips the scales massively towards the current major players.

Instead, I agree with the thesis of Andrew Ng and the AI Fund. The existing and emerging toolset around supervised learning and generative AI - the two most commercially useful paradigms of AI today - now provides the opportunity for founders who are subject matter experts in well scoped, hard / boring problems that will save companies money, time or both, to create great businesses with solid opportunities for growth and acquisition.

Andrew uses the example of Bearing.ai - a startup using AI to help save fuel costs in maritime shipping. Similarly, I’m impressed by Phaidra.ai, a closed loop AI control service that delivers step function improvements in plant stability and energy efficiency for energy-intensive industrial processes. And I’m intrigued by Ajust, working to resolve consumer complaints and improving the relationships between consumers and businesses, and RapidAIM, creating accurate pest forecasts and driving down the need for insecticide application.

The toolsets available today - from Open AI and others - have massively increased what’s possible for a small team with limited data and compute to achieve. However, to move at speed, you still need experience in end-to-end AI product design, evaluation and machine learning engineering and ops to run a responsible, reliable and cost effective product.

So I look for a well defined problem statement: something that users will pay to have removed from their list of frustrations. Substantial subject matter expertise in the core business problem to be solved. And substantial experience in designing and building AI systems at scale. Where those three are combined, I would be very interested in knowing more.

Check out Kendra’s Data Runs Deep Substack here:

---

That’s all! Feedback is very, very welcome.