© Krot_Studio/Shutterstock.com
Artificial intelligence (AI) startups are emerging at breakneck speed. But Writer claims to have a major advantage: it spends much less than the competition to train its large language models.
Synthetic data to train the 'IA
The young American company, founded in 2020, has just presented its latest model, Palmyra X 004. It spent around $700,000 to train it, including data and processors. It’s simply mind-boggling. For comparison, OpenAI spent nearly $100 million to train GPT-4, which powers ChatGPT.
To achieve this feat, Writer uses synthetic data to train its models, meaning that they are themselves generated by artificial intelligences. Its rivals, for their part, use the immense quantities of information available on the Internet.
A technique that, in addition to being expensive, has a significant limitation: according to a study carried out by AI researchers, if current development trends continue, companies “will completely exhaust” publicly available training data between 2026 and 2032. “Public textual data generated by humans will not be able to be developed beyond this decade ”, they explain. However, AI always needs more data to improve further.
© Unsplash/BoliviaInteligente
Beware of hallucinations
However, synthetic data also has some constraints. They are more likely to degrade model performance and exacerbate existing biasesdue to a greater propensity to hallucinate. Aware of the risks, Waseem Alshikh, co-founder and CTO of Writer, wants to reassure users:
200% Deposit Bonus up to €3,000 180% First Deposit Bonus up to $20,000“There is some confusion in the industry about what constitutes synthetic data. To be clear, we do not train our models on falsified data or hallucinations, and we do not use a model to generate random data. We take real, factual data and convert it into synthetic data that is specifically structured in a clearer and cleaner way for training models.”
The use of synthetic data is spreading rapidly, especially among giants like Meta, Amazon and Microsoft.
New fundraising in sight for Writer
Writer’s promoted training costs are appealing to investors, especially since the AI sector is struggling to reach profitability due to exorbitant operating costs. As such, the startup is working to raise up to $200 million from investors at a valuation of $1.9 billion, quadruple compared to September 2023.
Its models are marketed to companies, which use them to generate texts in a variety of contexts. This ranges from simple messages on LinkedIn to data analysis or the creation of custom AI applications. Writer has more than 250 client companies operating in various sectors, such as Accenture, Uber, Salesforce, and L’Oréal.
- The startup Writer trains its models with synthetic data, making its training costs lower than the competition.
- The synthetic data is itself generated by AI, and imitates the content produced by humans.
- While this practice is attractive in terms of price, it also maximizes the models' propensity to hallucinate.
📍 To not miss any Presse-citron news, follow us on Google News and WhatsApp.
[ ]