Show HN: Gentrace – connect to your LLM app code and run/eval it from a UI https://ift.tt/An7N6dv

Show HN: Gentrace – connect to your LLM app code and run/eval it from a UI Hey HN - Doug from Gentrace here. We originally launched Gentrace via Show HN in August of 2023. Since then, a million products have emerged in the LLM ops category. And what we've noticed is that almost none of them solve the core workflow: testing prompts, parameters, and other changes in your actual app, from a frontend where people can collaborate on the dataset, evals, or experiments to be run. So, we built that and are relaunching the company around that idea. Gentrace is the collaborative LLM app testing and experimentation platform that brings together engineers, PMs, subject matter experts, and more to run and test your actual end-to-end app. To do this, use our SDK to: - connect your app to Gentrace as a live runner over websocket (local) / via webhook (staging, prod) - wrap your parameters (eg prompt, model, top-k) so they become tunable knobs in the front end - edit the parameters and then run / evaluate the actual app code with datasets and evals in Gentrace We think it's great for tuning retrieval systems, upgrading models, and iterating on prompts. It's free to trial. Would love to hear your feedback / what you think. https://gentrace.ai/ December 11, 2024 at 02:05AM