The astounding capabilities displayed by large language models like GPT-4, Midjourney or Dall-E 3, created by leading AI labs, have sparked waves of interest and development in Generative AI and the transformational change it can bring to businesses. Following ChatGPT’s public release, the race to develop and scale up powerful language models has intensified. On one side are big tech firms and startups leveraging their vast resources to build proprietary models. On the other are open-source initiatives by academics and developers aimed at democratizing access to large language models.
As we look ahead, a key question arises - will proprietary models ultimately win out, or can open-source alternatives compete and even surpass them? The answer lies in a complex interplay between technological innovation, economic factors, applications and use cases, and community support.
The Rise of Proprietary AI
The most prominent proprietary Generative AI model today is OpenAI’s GPT-4. Built leveraging massive compute resources and datasets to achieve state-of-the-art performance on natural language tasks, it is the largest language model ever created.
GPT-4 displayed an unprecedented ability to parse complex language prompts and generate surprisingly coherent completions spanning several use cases. It answers questions, summarizes passages, translates text, and even generates code and poetry. The model represents a giant leap forward in AI's language mastery.
Though details remain sparse, it is more likely than not, that following GPT-4, OpenAI will keep working on developing even more advanced models across all modalities – images, video, voice, etc. Meanwhile, other tech companies are racing to stake their claim. Google has PaLM 2, and Anthropic has Claude, whereas Amazon has Titan. These firms leverage their extensive data and engineering talent to train massive proprietary models. Access is restricted through APIs and cloud platforms, allowing them to monetize the technology investments.
The core advantages of this proprietary approach are:
Flexible pricing (based on tokens generated) – easier to start with.
Seamless adjustment to computational resources – scales up or down per need.
Continued support for model development and updates – very important!
Easy Integration with existing services and infrastructure – via simple API calls.
Dedicated support and SLAs for business use-cases – someone to talk to.
Ability to customize models for specific domains and applications.
For large enterprises, these benefits make proprietary models highly appealing. The technology is polished and accessible without requiring in-house AI expertise. Plus, there’s support and someone to talk to. For high-value use cases like search, conversational AI, and content generation, proprietary models seem to currently lead the pack. Also, most of the vertical startups offering various tools on top of AI, are simply using a proprietary model such as OpenAI.
The Promise of Open-Source AI
In contrast to proprietary efforts, the open-source AI community aims to make large language models accessible to all.
Notable initiatives include Llama 2, Falcon 180B, and several others. These models are not locked behind paywalls or closed APIs. The code, datasets, and models themselves are publicly released for anyone to freely use, modify, and build upon.
By tapping into the creativity of decentralized developers and researchers worldwide, open-source models are rapidly evolving. The community scrutinizes the technology, conducts research, and identifies weaknesses. This transparency and peer-review helps build trust.
Importantly, open-source models are modular. Instead of monolithic architectures, they are composed of discrete components like vocabularies, encoders, and decoders that can be mixed and matched. This flexibility enables customization for different applications and languages.
Decentralized collaboration also leads to model diversity. Rather than a single approach, various techniques can be tried to push performance on different tasks. Over time, best practices surface and get integrated into newer architectures. For smaller organizations and individuals without access to huge compute resources, open-source models provide a pathway to utilize and innovate with large language models. The community-driven approach has resulted in remarkable innovations across the board. Transparency, innovation, and private data fine-tuning are key reasons why companies are looking at open-source models.
However, open-source models do face some challenges. Without big tech budgets, scaling a model to fit business needs requires clever optimizations and techniques – and sometimes this may not even be practical depending on the architecture of the model. Training datasets and benchmarks are also limited compared to what large firms can leverage. Continued community support across code, documentation, tools, and infrastructure is also vital so businesses start trusting them. These are also why most businesses still are debating using open-source models.
Thanks for reading Second Order AI! Subscribe for free and receive new posts that dissect the impact and innovating AI is creating for businesses worldwide.
Emerging Trends and Innovations
As both proprietary and open-source models continue evolving, we can expect multiple trends and innovations to shape the landscape:
Multimodal models:
Models that can process and generate image, video, speech, and text in an integrated fashion will expand capabilities into new modalities. They demand even more data and compute, playing into the strengths of big tech firms.
Specialization:
Rather than monolithic "generalist" models, we'll see more domain-specific models tailored for precise tasks and knowledge. These could emerge from both proprietary and open-source efforts.
Efficiency innovations:
Methods like sparse attention, pruning, knowledge distillation, and quantization will allow training and running larger models without proportional increases in compute. This will benefit open-source models.
Real-world grounding:
Models relying more on real-world knowledge graphs and causal models to make logical inferences, rather than just pattern recognition on text corpora will start to see traction over time. Companies with vast amounts of data will leverage what they have and build better models – eventually leading to MaaS – Models as a Service as one of the business models.
Reinforcement learning:
Self-play and simulation environments will likely train models to act dynamically and optimize for complex objectives over time. As this happens, models will be further democratized minimizing quality differences between open source and proprietary models. However, this doesn’t necessarily mean that businesses will start preferring open-source models – it could in fact be the other way around.
Hardware acceleration:
Specialized chips and architectures to massively speed up transformer-based models, reducing training compute needs. Although Nvidia is the current leader here, others are going to try and catch up with it and compete pretty soon.
As these innovations interact, we'll see both proprietary and open-source models advance via different paths. I believe that the strengths of each approach will drive success for different applications.
The Path Ahead
So where does this leave us? In the near term, proprietary models will likely continue to lead state-of-the-art performance as companies like OpenAI, Anthropic, Google and Microsoft dedicate enormous resources to developing ever-larger models and easier than ever access to those models, including for fine-tuning and private hosting.
However, open-source models are rapidly catching up in capabilities while also providing transparency, flexibility, and community support. As more talented researchers and developers get involved, they could out-innovate big tech on niche applications and non-English languages.
Ultimately, both proprietary and open-source models will co-exist serving different needs:
Proprietary for high-value enterprise applications where reliability, support, and integration matter more than transparency in model development.
Open source for smaller more focused models, richer fine-tuning, research, and use cases where cost needs to be controlled over time.
Both types of models currently support hosting in company’s private cloud via Azure, Amazon, or Google - so eventually, companies will stop debating about data security and privacy and move to use models where the costs and benefits workout the best. Proprietary models too, will likely simply offer a pricing model that is based on compute hours, instead of tokens.
The real winner will be the businesses who now have access to incredible Generative AI capabilities unimaginable just a few years back. As models continue to advance on both fronts, we'll see unprecedented opportunities for innovation across virtually every industry. The battle lines are drawn, and the race is on! Expect stunning breakthroughs in coming years that take Generative AI to new heights.
We can’t wait to see what the future holds.