Perplexity is an AI-based search engine that has been getting attention lately (especially in the US for now). At its core, it is a search engine, its engine is an AI model, and its interface borrows the chat format. What is different from traditional search engines or LLM-family tools (?) is that when a user asks a question in natural language, Perplexity AI combines information from the web with its trained data to generate the answer.
*For reference, the word perplexity is a metric used in NLP (natural language processing) to evaluate the performance of language models.
Looking at the model structure, it seems as if MOE (mixture of experts) has been applied (...is that right? This is close to my own guesswork). The system seems to combine performance improvements from fine-tuning OpenAI GPT-3.5 Turbo with the interfaces of LlaMa Chat (Meta) and Bird SQL (Twitter) and Microsoft Bing as the search engine. Through OpenAI Codex, natural language is converted to SQL so that even huge databases like Twitter can be searched easily.
The main features you can feel while using it are: 1) real-time web search, 2) summarizing the search results through interpretation using the internal knowledge accumulated via fine-tuning, 3) referencing the evidence for each interpretation (as a link), and 4) not stopping at just generating an answer but also suggesting follow-up questions - these are what set it apart from classic search engines and generative language models.
It looks like one of the most effective cases of applying the various APIs LangChain offers not just for research but for an actual prototype product.
