丨Highlight the key points
① Faced with the computing power and data advantages of large technology companies, smallAIStartups need to focus on building products and services that are different from large companies and have unique value.
② Data is often overvalued as a marketable asset. Its real value lies in how the data is used, not the data itself.
③ There is uncertainty in technological progress and market response, and the venture capital model accepts a certain proportion of failure as part of the innovation process.
④ Companies such as Google and Microsoft are willing to sacrifice national and global interests in pursuit of profits, while at the same time demanding that governments restrict the openness of technology, revealing the moral and strategic contradictions of the dark side of capitalism.
⑤ Major technological advances in history are often accompanied by financial bubbles, which is a natural part of the promotion of new technologies.
⑥ The development of the Internet has gone through a process from open to closed. The development of artificial intelligence may face a similar choice, which will have a profound impact on its popularization and innovation.
⑦ Although technological innovation may be accompanied by speculation and bubbles, it ultimately promotes social progress and economic growth and should be regarded as a positive social force.
Tencent Technology News According to foreign media reports, American venture capital companiesa16zA joint video was recently released on the video platform Youtube.FounderVideo by Marc Andreessen and Ben Horowitz. In this video, Andreessen and Horowitz discuss how small AI startups can compete with large technology companies that have advantages in computing power and data scale; reveal why data is overvalued as a saleable asset; and the similarities and differences between the AI boom and the Internet wave at the beginning of this century.
In 2009, Andreessen and Horowitz co-founded the venture capital firm Andreessen Horowitz.Venture CapitalThere are 16 letters between the first letter A and the last letter Z of the company name, so it is abbreviated as a16z. There is also a saying that as long as it is satisfied with the investment object, this venture capital company will participate in all financing of the startup company from A round to Z round.
The founder of the world's top venture capital a16z talks about AI andEntrepreneurshipFull video (AI subtitles, for reference only)
The following is the full text of the conversation:
Opportunities in the shadow of giants:How AI startups survive
Anderson: Today we are going to discuss a very hot topic - artificial intelligence. We will focus on the current state of the artificial intelligence industry as of April 2024. We hope that this will be helpful to anyone working in a startup or working in a large company. As always, we have solicited topics on the social media platform X and received many great questions, so we have prepared a series of audience Q&A, and now let's get straight to the point.
The first three questions all focus on the same theme: First, how do AI startups compete with big companies? In the face of the coming AI era, what should startup founders focus on building now? Second, how can small AI startups compete with established technology companies that have huge computing power and data scale advantages? Third, for startups that rely on technology from companies like OpenAI, which companies will benefit from future exponential improvements in the underlying models, and which companies may fail as a result?
I'll start with the first question and then go deeper. OpenAI CEO Sam Altman recently made some points in an interview that I agree with very much. His point is that as a founder of a startup, you should expect that the basic models launched by large AI companies will be greatly improved, and you should develop a response strategy in advance. If the current basic model performance is improved by 100 times, how will the founders of startups react? Should they be excited because it is good for their company; or should they be worried because it may bring a series of problems? What do you think about this?
Horowitz: I basically agree with Altman's point, but there are some details to note. From his perspective, he may be dissuading people from building their own base models, and I don't completely agree with this. Many startups that are building base models are actually doing very well. There are many reasons for this, the first of which is the difference in model architecture, which determines how smart the model is, how fast it responds, and how well it performs in a specific domain. This applies not only to text models, but also to image models. Different types of images react differently to prompt words. For example, if you ask the same question to two models, they will respond in very different ways depending on the use case.
Second, the emergence of model distillation (distillation, which can transfer the knowledge of large models to smaller and more efficient models, while maintaining performance while reducing computing power and memory requirements). OpenAI can develop the world's largest and smartest models, and startups can launch distilled versions of the models, achieving very high intelligence at a lower cost. Given this, while the models of large companies will undoubtedly become better, if the models built by startups are different in some aspects or focus on different areas, then even if the models of large companies are getting better and better, it will not necessarily affect these startups.
If a startup chooses to go head-on with big tech companies, it may encounter real problems because the latter have huge amounts of cash in their bank accounts. But if the startup is doing something unique enough or focusing on different areas, the situation will be completely different. For example, Databricks, a startup that recently launched the open source model DBRX, although the company also developed a basic model, it used this basic model in a very specific way with its leading data platform. Even if OpenAI's model becomes better, it is not enough to pose a real threat to AI models that focus on specific areas. The voice model of ElevenLabs, an AI voice cloning startup, is already embedded in everyone's AI stack. Everyone uses the company's voice model as part of the AI stack. In addition, ElevenLabs' voice model has a developer interface. This startup focuses on what it does. Although some startups appear to be competing with OpenAI, Google or Microsoft on the surface, there is actually no real competition, and I think such companies have broad prospects.
AI intelligence has its limits.Training data quality is critical
Anderson: Let's go a little deeper into the question of whether "God models" will be 100 times better. Do you think that big models, so-called "God models," will really be 100 times better?
Horowitz: I tend to think that it is possible for large language models to perform 100x better. Given what we know about large language models today, even though they are very advanced, the real difference may only be noticeable to professionals who study them in depth. If we are talking about a 100x performance improvement, then it stands to reason that we should be able to see a clear performance gap between some models and others. However, for ordinary users who use large language models in their daily lives (such as asking questions and getting information), this improvement may not be so obvious.
Anderson: The improvements we expect may include the breadth of knowledge and the improvement of capabilities. I think in some aspects, such as the speed of the model's response to different questions and the breadth of its knowledge, it is indeed possible to achieve significant improvements. In addition, the refinement and quality of the output will also be key to improvement. This includes reducing false information, that is, reducing "hallucinations", and ensuring that the answers are based on facts.
Horowitz: I agree that these aspects will improve significantly, because AI technology is developing rapidly in this direction. The challenge we face now is the alignment problem of the models, that is, although the models are getting smarter, they are not always able to accurately express the information they know. This alignment problem also limits the intelligent performance of the models to some extent.
Another question is whether we need a breakthrough to move from current AI, which I call “artificial human intelligence,” to a more advanced form of “artificial general intelligence.” By “artificial human intelligence,” I mean AI that has already reached an amazing level of mimicking human cognition and language use, and can perform many tasks that humans can. But to reach a broader level of intelligence, we may need some form of technological breakthrough.
If our current technology is close to the limit, it may not achieve a 100x improvement in some aspects. They are already pretty good compared to humans. Still, AI is expected to make huge leaps in knowledge mastery, hallucination reduction, and performance in multiple dimensions.
Anderson: There is a chart circulating in the industry. I can't remember its specific coordinate axes, but it roughly shows the performance improvement of different artificial intelligence models. In some tests, the scores of artificial intelligence models are only slightly higher than those of ordinary people. This is not surprising, because the training of artificial intelligence is entirely based on human data. Some people argue that these tests are too simple? Are more complex tests, like the SAT, needed to truly measure the ability of artificial intelligence? If many students get a perfect score of 800 in both math and language in the SAT, does this mean that the scoring criteria are too limited? Do we need a test that can truly test Einstein's level of wisdom? Existing testing methods certainly have their value, but we can imagine an SAT that can accurately distinguish people with super high IQs, a test that can truly measure the reasoning ability of artificial intelligence beyond human levels.
Horowitz: It is indeed true. AI may need such tests. In addition, another question that is often asked, and one that we have been discussing internally, is whether we need to take more provocative, more optimistic, or even more sci-fi predictions. When training a language model with data from the Internet, what is the nature of the Internet dataset? It is actually the average of everything, it is a representation of human activity. Due to the characteristics of the distribution of intelligence in the population, most content is at an average level, so the dataset used to train the model represents ordinary humans on average. Using such data, we can only train a very ordinary model. Most of the content on the Internet is created by ordinary people, so the content is ordinary overall, and the answers generated are also ordinary. By definition, the answers on the Internet are ordinary on average.
If we ask the model some routine questions like “Does the Earth go around the sun?” using the default prompts, we get an average answer, which is good enough. But here’s the key point: while the average data may come from an average person, the dataset also contains everything that all the smart people have written and thought, and all of that content is in the dataset. So this leads to how to guide the AI with specific prompts so that it can train with the “super genius” content in the dataset. If we construct the prompts differently, we can actually guide the AI down different paths in the dataset, resulting in different types of answers.
For example, if you ask an AI to write a piece of code that performs a task, like sorting a list or rendering an image, it will come up with ordinary code. But if you ask it to write secure code, it actually generates good code with fewer security vulnerabilities, which is very interesting. This is because it has access to a different training data set, a data set of secure code. Another example is that if you ask an AI to write an image generation program in the style of video game programmer John Carmack, you will get a much better result because it has access to code in the data set written by Carmack, who is one of the world's top graphics programmers. You can imagine that in many different fields, with carefully designed prompts, we can unlock the potential super genius of AI to get better answers than the default answer.
Anderson: I agree with this view. I think there may be a potential limit to the intelligence of artificial intelligence. We have discussed before that the world is very complex, and intelligence is about how to better understand, describe, and represent the world. But in the current iteration of artificial intelligence, humans build structures and then input the constructed structures into artificial intelligence, so artificial intelligence is good at predicting how humans build the world, rather than what the world actually is. The latter may be more complex and may not be simplified in calculation.
Are we going to hit a limit where AI can be very smart, but the limit is that it's as smart as the smartest human, not smarter than the smartest human? There's also the related question of whether AI can imagine completely new things, like new laws of physics and so on. Of course, there's probably only one in three billion humans who can do that, which is a very rare type of intelligence. So AIs are still extremely useful, but if they're artificial humans, rather than so-called superhumans, then they play a different role.
Horowitz: Take the super optimistic scenario of AI performance improving a hundredfold, a cynic would say that Altman claims that AI will be a hundredfold better. This is precisely because they can't get better. Altman basically said this for the purpose of intimidation, to prevent others from competing.
Anderson: I think whether AI is a hundred times better or not, Ultraman is claiming that this day will come. For those who don't know Ultraman, he is a very wise man. But there is no doubt that he is a genius with a strong competitive spirit. So before making any decision, you have to take that into account.
Horowitz: If AI performance increases are modest, Altman might say they will get better; but if they are indeed expected to improve significantly, he will also promote this angle. After all, why not? Now, let's explore the optimistic argument that AI performance may increase by a factor of 100 or even 1,000 and continue to rise over the long term. While each of the points I'm about to make may be controversial, many smart people in the field of AI support my views.
First, generalized learning is occurring inside neural networks, as confirmed by introspection techniques. These techniques allow us to observe how neural circuits inside neural networks evolve during training. Neural networks are developing general computational capabilities. For example, recently, a neural network was trained on a large number of chess games, and as a result, the network built a model of the chessboard and was able to make innovative moves. In addition, so-called "overtraining" - training the same model on the same data for longer - is actually proven to be effective. In particular, Meta and some other companies have recently been discussing how so-called overtraining actually works, which is basically continuing to train the same model, training it on the same data for longer, throwing more computing cycles at it. Some experts in the field even think that this approach works very well and there are no concerns about diminishing returns from additional training.
Anderson: Overtraining is the main technique used in Meta's recently released large language model Llama. In fact, some experts tell us that we may not need more data to improve the performance of these models at present, but more computing cycles. By significantly increasing the number of training times, the performance of artificial intelligence is expected to be significantly improved.
Horowitz: The role of supervised learning in improving artificial intelligence in data labeling cannot be underestimated. It has brought huge performance leaps for these models.
Anderson: We are currently hearing a lot of anecdotes and reports about AI self-improvement cycles that are actively underway. Many leading practitioners in the field believe that some form of self-improvement cycle is already in effect. This involves having the AI perform what is called a "chain of thoughts," where it solves a problem step by step until it becomes proficient at the process. However, by retraining the AI on these answers, a forklift-like upgrade can be achieved in each cycle of reasoning ability (i.e., users must replace old hardware with new equipment in order to gain the benefits of new technology).
Many experts believe that this self-improvement approach is now starting to work. There is also a discussion about synthetic data, and although there is still a heated debate about this, there is a considerable amount of optimism about it. Also, large language models may be good at writing code, but they may be better at verifying code. In fact, it would be a huge improvement if large language models were better at verifying code than writing code. This also means that AI may need to learn to verify the code they generate themselves, which they do have. We have an anthropomorphic bias towards AI systems, which can be misleading because we tend to think of the model as "it". Then we wonder how "it" can be better at verifying code than writing code. But AI is not just "it", it is a huge latent space, a huge neural network. In this network, writing code and verifying code may be composed of completely different parts, and there is no necessary consistency between the two.
If AI gets better at one of these things, then the part it's good at has the potential to make the part it's not good at better and better, too. That's the potential for self-improvement. In addition, there are many other factors driving the development of AI. For example, the current extreme shortage of chips limits the potential of many AI systems, but this limitation will be gradually lifted over time. There is also the issue of data annotation. A huge amount of data has been accumulated in AI systems, but there is still more data waiting to be mined around the world. At least in theory, some leading AI companies are investing money to generate new data; at the same time, the quality of even open source data sets is steadily improving. Therefore, we can expect significant improvements in the quality and quantity of data.
In addition, a lot of money is pouring into the field of artificial intelligence, providing strong support for the development of such technologies. At the same time, work on systems engineering is also going on in parallel, and many systems originally built by scientists are now joined by world-class engineers who are fine-tuning these systems to make them run more efficiently.
Horowitz: Maybe this is not an easy choice. This not only makes the inference process more efficient, but also improves the efficiency of the training process.
Anderson: Indeed, there are other areas of improvement. For example, Microsoft recently released its small language model, and there are reports that it can compete with larger models in performance. One of the key things Microsoft did was to optimize the training set. Specifically, they removed all duplicates in the training set and focused on training with a small amount of high-quality data, rather than relying on the large amount of low-quality data that most people use. Adding all of these factors together, we have about eight or ten different combinations of improvement directions, both practical and theoretical, and they are all being worked on in parallel. It seems hard to me to imagine that this combination of factors will not lead to significant improvements from the current state of the art.
Horowitz: I totally agree, it will definitely happen. Just going back to Altman's question or his proposal, if you start an AI startup and you think your technology will be as good as GPT-4 in two years, then you would consider not continuing. That would be a bad decision.
Anderson: This gets to the heart of what a lot of entrepreneurs worry about. The problem they're trying to solve is, I know how to leverage chat, I know how to build a SaaS app, and I know how to leverage a big language model to make great marketing materials. Let's say this is a really simple thing, and I built this whole system for it. Will it be that in 6 months, the big model will be able to do a much better job of making marketing materials with just simple prompt words, making my seemingly complex system irrelevant because the big model did it directly? Think of it this way, many of the current AI application companies are called "GPT wrappers," with a thin wrapper around ChatGPT that means ChatGPT can commoditize them or replace them.
Of course, the counter-argument is a bit like saying all old software applications are database wrappers. It turns out that wrappers around databases are like most modern software, a lot of that software ends up being very valuable, and there's a lot of stuff built around the core engine. How should an enterprise think about this when it thinks about building an application?
Horowitz: That's a really tricky question because there's a correctness gap. Like, why do we have Copilot instead of AI pilots? There are no AI pilots, there are AI co-pilots. It really comes down to the fact that we can't yet trust AI systems to be correct when they draw an image, write a program, or even write a court brief without making up a quote. All of these things require a human, and it's quite dangerous to not have a human involved. I think at the application level, in order for this to be really useful, you need to turn the co-pilot into a pilot. Can we do that? That's an interesting and difficult question. There's also the question of whether it's better done at the model level or at some layer on top? By doing things like using code validation to distill the correct answer, or is that just something that the model can do? I think that's an open question.
When you go into a domain, there may be packaging around things. I think one area where models are not very good is process. In the database world, just like the database analogy, there is one part of the task of writing briefs in a law firm, but there are 50 other tasks and things that need to be integrated into the way the company works, such as process, orchestration, etc. There may be a lot of these things, such as if you are making a video, there are a lot of tools, such as who will write the lyrics, which AI system writes the lyrics, which one produces the music, how all of these fit together, how to integrate them, etc. All of these things often require a real understanding of the end customer, etc. This is often what makes applications different from platforms in the past, because they have real knowledge about how the customer uses it, which is not related to the purpose of the platform design. It is very difficult for enterprises or individuals to extract this from the platform. So, I think these things may work, especially if the process is very complex, which is interesting.
Anderson: I often advise founders to think in terms of pricing strategy. That is, you can work backwards from the ultimate business value. In other words, first determine how much value your technology can bring to customers, and then set the price based on this value. Technology experts usually think about their technical capabilities first, then decide how to commercialize it and think about how to price it. They will try to find a balance between costs and what they think is a reasonable markup. If they have a monopoly in the market, they may set a higher markup. This is a technology supply-oriented pricing model. However, there is a completely different pricing strategy, namely the value-based pricing model, which focuses on the business value of the product or service to customers rather than simply the cost markup.
For example, if we are talking about a business opportunity worth $1 million, can we price it at 10%, which is $100,000? The logic of this pricing strategy is that if the product or service is worth $1 million to the customer, they may be willing to pay 10% to get it.
Horowitz: In our portfolio, there is an AI startup that focuses on services such as debt collection. Imagine if you could recover more debts with less manpower through a co-pilot-style solution, the value of this solution far exceeds the cost of purchasing an OpenAI license. Because the OpenAI license itself does not directly help with debt collection, and this startup's solution can significantly improve the efficiency of debt collectors. Therefore, the key is how to bridge the gap between the value of the product and the customer's willingness to pay. There is a very important point: an important criterion for testing whether your business idea is good is how much you can charge for it. Do you price it according to the value you bring to the customer, or just according to the amount of work the customer needs to put in? In my opinion, this is the real touchstone to measure the depth and importance of the value you create.
Anderson: Indeed, there are some types of businesses that are difficult to accurately value from a technology investor's perspective. These businesses provide specific solutions to business problems that are of great value to customers, so they are willing to pay a significant portion of that value as a fee. This model means that even if the business is not very differentiated in technology, they can still be very profitable. And because of the profitability of these businesses, they can actually afford to think deeply about how technology fits into the business and what else they can do. For example, the success story of Salesforce.com is like this.
Horowitz: There is also a view that as the performance of all models is improving rapidly and there are excellent open source models such as Llama and Mistral available, the real value creation will be in the orchestration and integration of tools, because you can simply plug in the best model at the time. And the models themselves will compete with each other in the market and may eventually become commoditized, and the lowest cost model will win. Therefore, some people believe that the best strategy is to bring powerful AI technology to the people.
The truth behind the speculative boom:The reality and fantasy of AI investment
Anderson: Let's move on to other questions. Some people ask, "Why do venture capital firms continue to invest heavily in AIGC startups when they know that such companies will not be profitable in the near future?" Others ask, "If AI reduces the cost of starting a startup, how will the structure of technology investment change?" Basically, these two questions are completely opposite. If you look at it blurry from your left eye, you will see that the capital invested in basic model companies is growing at an astonishing rate. Such startups are raising hundreds of millions, billions, and even trillions of dollars, which makes people marvel at the scale of capital in these companies. But if you look at it blurry from your right eye, you will see that it is much easier to build software now, and it is much easier to own a software company or have a small group of programmers write complex software because they are supported by AI assistants and automated software development capabilities. Therefore, the cost of starting an AI application startup may drop significantly, and the starting capital may only need one-tenth, one-hundredth, or one-thousandth of the cost of starting Salesforce.com. How do we look at this duality, that from either perspective, we can see that the cost will either soar or actually approach zero.
Horowitz: We do have investments in both types of startups. I think the companies that have historically achieved profitability the fastest are the AI companies whose revenues have grown far faster than their costs. In our portfolio, the AI startups have all grown their revenues quite fast, actually faster than their costs. At the same time, the startups that have raised hundreds of millions or even billions of dollars developing foundational models are also generating revenue very quickly, and all of these companies have very few employees. So I would say that even a startup like OpenAI, in terms of the number of employees relative to their revenue, they are not a large company. If you look at their revenue levels and how quickly they got there, the number of employees is quite small. Of course, the total expenses of such startups are huge, but they are all invested in the creation of the models. This is an interesting thing, and I'm not sure how to fully understand it, but I think that as long as you are not building foundational models, it will make the startups more efficient and potentially reach profitability faster.
Anderson: There is a very optimistic counter-argument that the falling cost of building new software companies may be an illusion. The reason behind this is a phenomenon in economics called the Jevons Paradox. The Giffen Paradox occurs when technological progress increases the efficiency of resource use, thereby reducing the amount of resource needed for any single use, but the falling cost of the resource stimulates an increase in demand that is elastic enough to increase rather than decrease the overall use of the resource. We can see examples of this phenomenon, for example on highways, where it actually increases traffic congestion. The reason is basically that with more roads, more people can live on them; businesses can expand, and as a result, there are more cars than ever before, and traffic becomes worse. Or think of the classic example of coal consumption during the Industrial Revolution. As the price of coal fell, people actually used more coal than ever before. Even though people got more power, the result was that more coal was used.
The paradox here is that while the cost of developing any particular piece of software has gone down, the response to that is a surge in demand for the software’s capabilities, so while it looks like the starting price for software companies will go down, what will actually happen is that because the software is able to do so much more, the quality of the product will go up so much, the product roadmap will be so exciting, the customers will be so happy that they want more, so the result will be that the cost of development actually goes up. Take Hollywood, for example, CGI (computer generated imagery) should theoretically have reduced the cost of making a movie, but in reality it has increased it because audience expectations have gone up. Watch a Hollywood movie now, it’s full of CGI. As a result, movies cost more to make than ever before. The result, at least in Hollywood, is that movies are more visually sophisticated, whether they’re better is another question, but they’re more visually sophisticated, more attractive, and stunning.
In software, because end users have access to better software, this makes them want more software, which in turn leads to higher software development costs.
Horowitz: Consider a simple example like travel. Booking travel through Expedia is complex, with users clicking through different screens and a high probability of making mistakes. The AI version of a booking system would be: send me to Paris; put me in my favorite hotel; arrange for me on the best airline at the best price; make this trip very special for me. You can also make the services provided by AI more complex. For example: we know this person likes chocolate, we will send the world's best chocolate from Switzerland to this hotel in Paris by FedEx, etc. It can be said that AI can provide services to a level that we cannot even imagine today. It is not possible at present, simply because software tools are not what they will become.
Anderson: I couldn't agree more with you. Consider this scenario: I arrive in Boston at 6pm expecting to have dinner with a group of very attractive people.
Horowitz: Indeed, such an arrangement is not available at any travel agency at the moment. Of course, you don’t necessarily want a travel agency to get involved.
Anderson: But you will gradually realize that this personalized experience needs to be closely integrated with my personal artificial intelligence. The infinite possibilities of creativity and the human ability to create new things have always been underestimated. This ability seems to be endless. John Maynard Keynes, a prominent economist in the first half of the 20th century, predicted that with the spread of automation, people would no longer need to work 40 hours a week. Keynes believed that once basic life needs - such as food and shelter - were met, people would no longer need to work to survive. But as time goes by, people's needs continue to increase, from refrigerators, cars, televisions to enjoying vacations, and the needs seem endless. I can't predict what we will need in the future, but what is certain is that there will always be someone who can foresee and create new needs, and these needs will soon become very attractive.
Horowitz: Keynes made this point in his book Economic Prospects of Our Grandchildren. Similarly, Karl Marx expressed a similar view. He believed that after the socialist utopia is realized, society will be able to regulate overall production, so that individuals can freely engage in various activities. Marx once described: "Hunting in the morning, fishing in the afternoon, raising cattle in the evening, and commanding the country after dinner." This picture of life depicts an ideal state of life. If I had to list four activities that I would not like to do, they would be hunting, fishing, raising cattle, and commanding the country.
What Keynes and Marx have in common is that they both had a very limited view of what people want to do. Moreover, people want to have a sense of purpose, to have a career, to pursue a goal, to be useful, to play an active role in life. This finding is somewhat surprising. Therefore, I have always believed that the demand for software is basically completely elastic and may grow indefinitely. As the cost of software continues to decrease, the demand will also grow. Because in the field of software, there are always new things to be done, and there is always room for new automation, optimization, and improvement. Once the current constraints are removed, people will imagine new possibilities.
Take artificial intelligence as an example. Currently, some companies are developing artificial intelligence security camera systems with advanced functions. In the past, software that can process and store video streams from different cameras and provide a playback interface has been considered a major technological advancement. However, modern artificial intelligence security cameras are already able to have an actual semantic understanding of what is happening in the environment. They can identify specific people and judge whether there are abnormalities based on their behavior and expressions. For example, the system can recognize that someone carries a gun because he has a habit of hunting; while other people who do not usually carry guns, if they suddenly carry a gun and show an angry expression, the system will judge that there may be danger. This kind of security system with semantic understanding is obviously much more complicated than traditional security systems, and its manufacturing cost may also be higher.
Horowitz: Imagine that in the healthcare field, we can do a comprehensive self-diagnosis every morning when we wake up to understand our health status. For example, we can ask: "How do I feel today? What are the levels of all my physiological indicators?" and "How should I interpret this data?" In the field of medical diagnosis, artificial intelligence is particularly good at handling high-dimensional data problems. When we can obtain data such as continuous blood sugar readings and occasional blood sequencing, we can gain deep insights into personal health status. The pursuit of a healthier life is everyone's vision. At present, our daily health monitoring methods, such as weight weighing or heart rate monitoring, are very primitive compared to the advanced health monitoring that may be achieved in the future.
The true value of data assets:The potential and limitations of AI models
Anderson: Now let's move on to a new topic. Regarding the discussion on data, someone raised the question: As AI models develop, they allow us to replicate existing application functionality at a very low cost, which makes proprietary data seem to become the most valuable asset. In your opinion, how will this affect the value of proprietary data? In this emerging environment, what other types of assets should companies focus on building? Someone else asked: In the new era of artificial intelligence, how should companies protect sensitive data, trade secrets, proprietary data, and personal privacy?
To kick off this discussion, I’ll start with a potentially controversial idea: “data is the new oil.” This idea holds that data is a key input to training AI and powering its operations. As a result, data becomes a new, limited, and extremely valuable resource. This is particularly true in the context of AI training. When exploring how to leverage AI, many companies often emphasize the proprietary data they have. For example, a hospital, insurance company, or other type of business might claim that they have a large amount of proprietary data that they can use in conjunction with AI to create results that others cannot easily achieve.
The point I am making is that in almost all cases, this is not true. It is really just a mimetic phenomenon. The amount of data available on the internet and in the broader environment is so massive that while there may not be an individual’s specific medical information, the amount of medical information I can get from the internet for many different situations for many people is so massive that it overwhelmingly outweighs the value of so-called “personal data.” So for social media platform X, its proprietary data may be slightly useful in some ways, but it is not actually going to have a significant impact and in most cases will not be a key competitive point.
The evidence supporting my view is that we have not seen a rich or mature data market so far. In fact, there is no large data trading market. Instead, we see some smaller data processing markets, such as data brokers, who sell a large number of Internet users' information to customers, but the scale of these businesses is relatively small. If data really has great value, it will have a clear market price and we will see it traded in the market. However, we do not see this, which in some ways proves that the value of data is not as high as people think.
Horowitz: I agree with that view. The value of raw data - those data sets that have not been processed in any way - is often overstated. I completely agree with that. Although I can imagine some exceptions, such as some special population genetic databases, which may be difficult to access and have unique value in certain research areas, these data are not just found on the Internet. I can imagine that these data are highly structured, general, and not easily and widely available. However, this is not the case for most data held by companies. These data are either widely available or specific and not general.
Nonetheless, some companies have successfully used data to improve business outcomes. Meta, for example, has used its data to great effect, by feeding it into its own AI systems to optimize its products in amazing ways. I believe that almost every company can enhance its market competitiveness by using its own data. However, it is actually unrealistic to think that once a company collects some data, it can monetize it like selling oil, or that this data is the new oil resource. Interestingly, a lot of the data that we think is most valuable, such as the company's own code base - that is, the software written by the company, a lot of it is stored on GitHub. As far as I know, no company we have worked with is building a separate programming model based on its own code, or whether this is a good idea, it may not be necessary because there is already a lot of code that exists and the system has been fully trained on this code. Therefore, it is not a significant advantage. I think only very specific types of data have real value.
Anderson: Let’s translate this into concrete actionable insights. If I manage a large company like an insurance company, a bank, a hospital group, or a consumer products company like PepsiCo, how can I verify that I actually have a valuable proprietary data asset and that I should focus on leveraging it, rather than investing all my efforts into trying to optimize the use of that data or completely switch to building solutions using Internet data?
Horowitz: Take the insurance business, for example. If you are in that industry, then all your actuarial data is interesting and relevant. I am not sure if anyone has publicly released their actual actuarial data. Therefore, I am not sure how you can train a model with just data from the internet. This is a good question to explore.
Anderson: Let me pose a challenging hypothesis: suppose I am an insurance company and have records on 10 million people, including their actuarial tables, when they got sick and died, etc. This sounds great. However, there is already a lot of general actuarial data about large populations on the Internet, because governments collect this data, process it, and publish reports, as well as data from numerous third parties and academic studies. Does your large dataset provide you with any additional actuarial information that is not already available in the larger datasets on the Internet? Are your insurance customers really actuarially different from the general population? These questions are worth pondering for every data manager.
Horowitz: I agree with this view: In the insurance business, when a customer applies for insurance, the company usually requires health checks such as blood tests to obtain information about the customer's health status. This information includes lifestyle habits such as whether the customer smokes, which are important data for insurance companies. However, in general databases, although you can know who has died, it is often unclear about the specific health conditions and lifestyle habits of these people before. Therefore, what insurance companies are really looking for is how long the life expectancy is for people with specific profiles and laboratory results, which is exactly where the value of data lies.
What's interesting is that a company like Coinbase has an extremely valuable asset in preventing illegal break-ins. They have put a lot of work into this and have certainly accumulated a lot of data about various break-in attempts. This data is likely very specific to the types of people who are trying to illegally break into cryptocurrency exchanges, so this data could be very useful to Coinbase. I don't think they would be able to sell this data to anyone, but I think if every company could feed the data into a smart system, it would help their business. I think there are very few companies that have data that can be easily sold.
At the same time, there is also the middle question of what data you are willing to let companies like Microsoft, Google, or OpenAI have access to. I think what companies are struggling with is not whether we should sell our data, but whether we should train our own models to maximize value, or whether we should feed our data into large models. If we feed our data into large models, will all our competitors have access to the information we just fed into them? Can we trust large companies not to exploit our data? If the competitiveness of the company depends on it, then probably it shouldn't be done.
Anderson: Yes, there are at least reports that some large companies are using all kinds of data that they shouldn't be using to train their models.
Horowitz: I think there's a good chance that these reports are true. Or, they should open up the data. We've talked about this before, companies that claim they're not stealing people's data or taking data in an unauthorized way, but refuse to disclose where their data comes from. Why don't you tell us where your data comes from? In fact, they're trying to shut down all openness - no open source, no open weights, no open data, nothing open, and trying to get the government to do that. If you're not a thief, then why would you do that?
Anderson: Exactly. To give you an example from the insurance space, it is illegal to use genetic data for insurance purposes. There is a law in the United States called the Genetic Information Nondiscrimination Act of 2008, which basically prohibits health insurance companies in the United States from using genetic data for health assessments. By the way, because genomics is becoming so advanced, this data, if it is actually used to predict when people are going to get sick and die, could be some of the most accurate data out there. And they are actually prohibited from using this data.
Horowitz: I think this is an interesting example of a weird misuse of good intentions in policy making that could potentially cause more deaths than all the health policies of the FDA and so on have saved. In the field of artificial intelligence, access to data about why everyone is sick, their genetic information, and so on, is the most valuable resource. It's like the "new oil." If we could match this information up, we would never know why someone is sick and could make everyone healthier. But to stop insurance companies from overcharging people who are more likely to die, we've actually locked up all of this data. It would be better if we massively subsidized health care for those who are more likely to need it, just as we normally do for individuals, and then solved the problem, rather than locking up all the data.
Anderson: One interesting question about insurance is, if you have perfectly predictive information about individual outcomes, does the whole concept of insurance still work? Because the whole theory of insurance is risk pooling. You don't know what's going to happen in individual cases, which means you build these statistical models and then you pool the risk, and then you have variable payouts based on what happens. But if you know exactly what's going to happen in each case because you have all this predictive genomic data, then suddenly risk pooling doesn't make sense anymore. You can just say, this person is going to spend X, that person is going to spend Y.
Horowitz: The current health insurance system does not seem to be fundamentally rational. The concept of insurance originated from agricultural insurance, where farmers jointly contributed to a fund to compensate individuals when their crops failed. This mechanism is designed to deal with catastrophic, unlikely events. However, in reality, everyone needs medical treatment, and people's health conditions vary, but health insurance operates through a complex system that increases costs, bureaucracy, and large companies. If our goal is to pay for people's health, why not pay directly for health care services? In addition, if we want to prevent people from over-seeking medical treatment, raising deductibles seems to be a more direct way.
Anderson: From the perspective of justice and fairness, if I know that someone's medical expenses will be higher than others, should I pay more for them? If we can accurately predict individual medical costs, or have an efficient prediction model, the willingness of members of society to share medical expenses may be significantly reduced.
Horowitz: Genetic factors are beyond personal control, but personal behavior can significantly increase the risk of disease. Therefore, we may be able to encourage people to stay healthy through incentives, rather than just paying medical expenses to prolong life. In my opinion, the current medical insurance system in the United States is extremely unreasonable and there are many areas that can be improved. Compared with some other countries, the US system is ridiculous.
From the Internet Bubble to the AI Boom:Will history repeat itself?
Anderson: Someone asked what the most significant similarities are between the current development of artificial intelligence and the Internet 1.0 era. I have a theory for this. Because of my role in the early days of the Internet and your experience at Netscape, we face this question often. The rise of the Internet is a major event in the history of technology and is still remembered by many people. People tend to reason by analogy, thinking that the rise of artificial intelligence must be similar to the boom of the Internet, and starting an AI company may be similar to starting an Internet company. However, we actually receive many similar questions, and they are all about analogies. Personally, I think this analogy is largely inapplicable. Although it does work in some ways, it does not work in most cases. The reason is that the Internet is a network system, while artificial intelligence is a computer-based technology.
Horowitz：That’s true.
Anderson: To help us understand this, we can compare the current discussion to the rise of the personal computer or, more appropriately, the development of the microprocessor. My analogy goes back to the era of mainframes. The remarkable achievement of the Internet is that it is a network that connects many existing computers together. Of course, people have also designed various new computers to connect to the Internet. But the key point is that the Internet is essentially a network of connections. This is crucial because most of the dynamics of the Internet industry, the competitive landscape, and the activity of startups are related to building networks or developing network applications. Startups in the Internet era are deeply influenced by network effects, and the positive feedback loops that occur when you connect a large number of people, such as Metcalfe's law describing that the value of the network increases as the number of users increases.
Although network effects exist in AI, they are more like a microprocessor or chip, that is, a computer. It is a system that takes data in, processes it, and outputs it, and then produces results. This is a new type of computer - a probabilistic computer, a computer based on neural networks. It does not always give precise results, and sometimes it may even argue with you and refuse to answer your questions.
Horowitz: This new type of computer is fundamentally different from traditional computers and is able to build complex things in a more compressed form. It has new and different and valuable capabilities because it can understand language and images and do tasks that previous deterministic computers could not solve. I think the analogies and lessons learned from the early days of the computer industry or the early days of the microprocessor are more relevant than the early days of the Internet. This does not mean that there will not be boom and bust cycles in the development of technology, because people's expectations of technology often go through cycles from over-excitement to over-pessimism. We may see over-investment in things like chips and energy, but I think the nature of the network and the development path of the computer are fundamentally different in the way they evolve, and their adoption curve will be different.
Anderson: This leads me to think about the future direction of the industry. A key question we face is: How will the industry develop? Will there be a few large models, or will there be a large number of models of different sizes? The history of the computer industry, especially the original IBM mainframe era, provides us with insights. These large computers were large and expensive and were limited in number. The general view at the time was that the world only needed a few computers. Thomas Watson, the founder of IBM, even thought that the world might only need five computers. This view was based on the needs of governments and a few large insurance companies. However, as the computer industry has experienced 50 years of development, we have seen the diversification and popularization of computer technology.
Horowitz: Yes. Who else needs to do so much math?
Anderson: Who needs to keep track of all those numbers? Who needs that level of computing power? Such needs may not seem relevant today. Incidentally, early computers were not only large but also expensive, making them a luxury that only a few could afford. In addition to the purchase cost, there was also the cost of the people needed to maintain these computers. In those days, computers were so large that entire buildings had to be built specifically for them. You would see people in white lab coats who looked after these computers because they had to be kept extremely clean or they would stop working. This scenario gave rise to the so-called 'AI God model', the concept of a large base model, and the idea of a 'God mainframe', where only a few such devices existed. If you've seen old sci-fi movies, there's usually a setting where there's a large supercomputer that's either doing the right thing or it's doing the wrong thing, and if it's doing the wrong thing, the plot of the movie is usually centered around how to fix or defeat it. This single, top-down concept persisted for the first few decades.
As computers started to get smaller, we entered the era of so-called “minicomputers,” which was the next phase. Minicomputers, which cost $50 million to $500,000, were still too expensive for the average person to afford. As a result, only medium-sized companies could afford them, and the average household could not afford them. The advent of personal computers then further reduced the cost to around $2,500, and the advent of smartphones further reduced the cost to around $500. Today, computing devices are ubiquitous, and they come in all shapes, sizes, and functions, and some cost as little as a penny, such as the embedded ARM chip in the thermostat that controls the temperature in a room. There are billions of these devices around the world, and a new car may contain as many as 200 computing devices, or more. Now, we generally assume that almost all devices have chips inside them, and that these chips require electricity or batteries to power them. In addition, we increasingly assume that these devices are connected to the Internet, because almost all computers are assumed to either already be on the Internet or will soon be connected to the Internet.
Therefore, today's computer industry forms a huge pyramid structure. At the top of the pyramid, there are still a small number of supercomputer clusters or large mainframes, which are like God models or God mainframes. Next are a larger number of small computers, personal computers, smartphones, and a huge number of embedded systems. The computer industry includes all these different types of devices. Depending on your needs, identity, and purpose, you can choose computers of different sizes. If this analogy holds, then it basically means that we will have AI models of various sizes, shapes, and capabilities, which will be trained on different types of data, run at different scales, and have different privacy policies and security policies. You will face huge diversity and changes, forming a complete ecosystem, not just dominated by a few companies. Yes, I would love to hear your thoughts on this.
Horowitz: I think that's a valid point. Also, I think another interesting thing about computing in this era is that if we look back at the entire era of computing from mainframes to smartphones, the difficulty of operating the devices has led to a lock-in of users. As the saying goes, "No one ever got fired for buying an IBM machine," and that's because people were already trained and had the skills to use the operating system. Given the sheer complexity of dealing with computers, it makes sense to choose a known and secure option. Even in the smartphone space, for example, Apple's smartphones, they have been able to dominate in part because it's very expensive and complicated to switch to other systems.
This brings up an interesting question about AI. Because AI is by far the easiest computer system to use, able to communicate in English just like a person. So what kind of lock-in is there here? Do users have complete freedom to choose the size, price, performance, and speed of the computer they need for their specific task? Or, are they locked into some "God model"? I think this is still an unsolved question, but it is very fascinating and will likely be very different from previous generations of computer systems.
Anderson: That makes sense. To finish up on that question, what lessons do you think we've learned from the internet age that we've been through are relevant and what factors should people take into account?
Horowitz: I think one of the big lessons is the cyclical nature of the dot-com era. You can see that the demand for the internet and the awareness of its potential was so high that money poured in like a torrential downpour. In the dot-com era, basic telecom infrastructure and things like fiber received unlimited funding, which led to overbuilding of fiber. Eventually, we had a fiber glut and many telecom companies went bankrupt, and although that brought some fun, we ended up in a good place. I think this situation is likely to repeat itself in the field of artificial intelligence, where every company may be funded, but we don't actually need that many AI companies, so many companies will fail and there will be huge investor losses. It is certain that the overproduction of chips will happen at some point, we will have too many chips, and some chip companies will definitely go bankrupt. The same situation may also happen with data centers, etc., which will lag behind and then overbuild, which will be very interesting. This is the natural law of the development of every new technology. The development of new technologies often goes through a cycle of overbuilding, underbuilding, and then overbuilding. In this process, the influx of money drives the construction of infrastructure, and although a lot of money will be lost, we get infrastructure because it is truly adopted and changes the world.
Another important aspect of the development of the internet is that it went through several phases. In the beginning, the internet was very open, and that was probably the single greatest boom that was ever good for the economy, and it certainly created tremendous growth and power, and it increased the economic power and cultural influence of the United States. Then, as the next generation of architectures came about, the internet became closed, and the discovery of the internet was almost entirely controlled by companies like Google. I think AI could go in two directions: it could be very open, or with the wrong regulation, we could force ourselves to go from technology that is open source, that anyone can build, to technology that only the companies that own the internet today own, which would put us at a huge disadvantage in competing with countries like China. So I think that's something that we're trying to make sure doesn't happen right now, but it's a real possibility right now.
Anderson: There is a real irony in that networks were once completely proprietary, but then they opened up.
Horowitz: Yes, that's right, AppleTalk, NetBEUI, and NetBIOS were all proprietary networks provided by specific vendors in the early days.
Anderson: Then the Internet came along, and with the development of technologies like TCP/IP, everything became open. But the field of AI seems to be moving in the other direction: big companies are trying to take AI in the opposite direction. AI started out open, like basic research. But now, they are trying to close it off, which is a pretty bad turn.
Horowitz:Yes, very bad. To me, it's really shocking. This is in some ways the darkest side of capitalism, when a company is so greedy that they're willing to destroy countries and even the world just to make a little extra profit. But the way they do it is very bad: they claim it's for safety, they claim we've created an AI that can't be controlled, but we're not going to stop developing it, we're going to keep building it as fast as we can, we're going to buy every GPU on the planet. But we need the government to step in and stop it from being opened up. That's exactly the position that Google and Microsoft are taking right now, and it's crazy.
Anderson:We will not protect it.
Horowitz:This has nothing to do with safety, it has everything to do with monopoly.
Anderson:Yes, and in relation to the speculation you mentioned, there is a criticism we hear a lot, which basically says, "You entrepreneurs, you investors are so stupid, every time a new technology comes along there's a speculative bubble, why don't you learn your lesson?" Well, there's an old joke that relates to this, which is that the four most dangerous words in investing are "this time is different." So, will history repeat itself? Will it not? It's undeniable that almost every major technological advance in history has been accompanied by some kind of financial bubble, basically since the existence of financial markets. This includes everything from radio to television to railroads. By the way, there was actually an electronics boom and bust cycle in the 60s called 'Tronics', where every company had 'Tronics' in its name. There was also the laser boom and bust cycle, all of these cycles.
Basically, any new technology, what economists call a "general purpose technology," which is something that can be used in many different ways, inspires a kind of speculative frenzy. And the criticism is, "Why do you need to have this speculative frenzy? Why do you need to have this cycle? Because you know, some people invest in something, they lose a lot of money, and then there's this bubble-bust cycle that makes everyone depressed. It can delay the adoption of the technology." There are two things to point out: One, if you're dealing with a general purpose technology, a technology like artificial intelligence that may be useful in many ways, no one actually knows in advance what the successful use cases or successful companies are going to be. You have to learn by doing, and you're going to have failures, and that's part of venture capital. Yes, we do. So, the real model of venture capital has this in mind to some extent. In core venture capital, the kind that we do, we basically assume that half of the companies will fail, half of the projects will fail. If you completely fail, like lose money, that's a problem.
Horowitz:Total failure is indeed like losing money.
Anderson:Of course, if we or any of our competitors could figure out how to do only the things that 50% works for and avoid the things that 50% doesn't work for, we would do it. But the reality is, we've been doing this for 60 years and nobody has figured out how to do it. So there's unpredictability. And, another interesting way to think about it is that if a society doesn't speculate on the emergence of new technologies, it means that the society is basically pessimistic about the prospects of new technologies and about entrepreneurship.
There are many societies like this in the world that simply lack the spirit of invention and risk-taking that places like Silicon Valley have. Are they better or worse? Generally speaking, they are worse. They are less focused on the future, less focused on creating things, and less focused on how to achieve growth. So, I think this is "inevitable phenomenon". Of course we want to avoid the negative impact of speculative boom and bust cycles, but it seems that every time a new technology emerges, it comes with this, and at least as far as I know, no society has found a way to get all the benefits without experiencing the downside.
Horowitz:Why wouldn't you want to? I mean, it's kind of like, you know, the whole American West was built on the gold rush, and the depiction of the gold rush in pop culture tends to focus on the people who didn't make a lot of money. But in reality, some people did make a lot of money and found gold. During the dot-com bubble, you know, this is mocked by every movie, if you look back at any movie between 2001 and 2004, they're all about how only fools would do this or that, and there were a lot of interesting documentaries and so on. But during that bubble, companies like Amazon, eBay, and Google started their businesses. These companies were founded during that time of great speculation, and there was really 'gold' in these companies. If you invested in any of those companies, you might have invested in the next one, including Facebook, Snap, etc.
So that's what it is, that's what's exciting about it. You know, moving money from people who are well-funded to people who are trying to do new things and make the world a better place is the best thing in the world. If some of us lose some of the excess funding in the process of trying to make the world a better place, why would you be angry about that? That's something I will never understand. Why be angry at young ambitious people who are getting funding to try to improve the world?
Anderson:Indeed, do we prefer to spend our money on just the material things like houses, yachts, and planes, compared to other things in the world, especially compared to other people who have a lot of wealth? How is that similar to what we're talking about now? Yes, just like the people we see in the news. We've answered four questions, and we're going very well, and we're doing great. Thank you all for joining us today.