Open HN: Helicone.ai (YC W23) – Initiate-provide logging for OpenAI

Hey HN – Justin, Scott, and Barak here. We’re wrathful to introduce Helicone (https://www.helicone.ai) an originate-provide logging solution for OpenAi purposes. Helicone’s one-line integration logs the prompts, completions, latencies, and charges of your OpenAI requests. It for the time being works with GPT, and is seemingly to be constructed-in with one line of code. There’s a demo at https://www.helicone.ai/video.

Helicone’s core technology is a proxy that routes all of your OpenAI requests through our edge-deployed Cloudflare Staff. These workers are incredibly generous and pickle off no discernible latency impression in manufacturing environments. As a proxy, we provide bigger than ethical observability: we present caching and steered formatting, and we’ll almost today add particular person fee limiting and model provider attend off to present sure that your app is mute up when OpenAI is down.

Our web application then provides insights into key metrics, equivalent to which users are disproportionately driving charges and what’s the token utilization broken down by prompts. That you just would possibly want to filter this recordsdata in accordance with personalized common sense and export it to different locations.

Getting started with Helicone is almost today and simple, no subject the OpenAI SDK you utilize. Our proxy-based mostly solution doesn’t require a third celebration kit—merely trade your request of’s depraved URL from https://api.openai.com/v1 to https://oai.hconeai.com/v1. Helicone is seemingly to be constructed-in with LangChain, LLama Index, and all different OpenAI native libraries. (https://medical doctors.helicone.ai/quickstart/integrate-in-one-line-of…)

We now enjoy got thrilling restful aspects bobbing up, one of which is an API to log particular person feedback. As an illustration, need to you’re rising a instrument admire GitHub Copilot, that you just can log when a particular person licensed or rejected an offer. Helicone will then aggregate your result quality into metrics and produce finetuning solutions for once that you just can save charges or give a boost to efficiency.

Sooner than launching Helicone, we developed several projects with GPT-3, including airapbattle.com, tabletalk.ai, and dreamsubmarine.com. For every project, we feeble a beta model of Helicone which gave us instantaneous visibility into particular person engagement and result quality disorders. As we talked to extra builders and companies, we realized they had been spending too great time building in-condominium alternatives admire this and that gift analytics products had been no longer tailor-made to inference endpoints admire GPT-3.

Helicone is developed below the Overall Clause V1.0 w/ Apache 2.0 license in say that that you just can employ Helicone interior your individual infrastructure. In case you fabricate no longer are looking to self-host, we present a hosted solution with 1k requests free per month to steal a survey at our product. In case you exceed that we provide a paid subscription as successfully, and also that you just can survey our pricing at https://www.helicone.ai/pricing.

We’re contented to introduce Helicone to the HackerNews neighborhood and would rob to hear your solutions, solutions, and experiences related to LLM logging and analytics. We’re wanting to steal in meaningful discussions, so please don’t hesitate to piece your insights and feedback with us!