Show HN: I built an OSS alternative to Azure OpenAI services https://ift.tt/ymAOdjp

Show HN: I built an OSS alternative to Azure OpenAI services Hey HN, I am proud to show you guys that I have built an open source alternative to Azure OpenAI services. Azure OpenAI services was born out of companies needing enhanced security and access control for using different GPT models. I want to build an OSS version of Azure OpenAI services that people could self host in their own infrastructure. "How can I track LLM spend per API key?" "Can I create a development OpenAI API key with limited access for Bob?" "Can I see my LLM spend breakdown by models and endpoints?" "Can I create 100 OpenAI API keys that my students could use in a classroom setting?" These are questions that BricksLLM helps you answer. BricksLLM is an API gateway that let you create API keys with rate limit, cost control and ttl that could be used to access all OpenAI and Anthropic endpoints with out of box analytics. When I first started building with OpenAI APIs, I was constantly worried about API keys being comprised since vanilla OpenAI API keys would grant you unlimited access to all of their models. There are stories of people losing thousands of dollars and the existence of a black market for stolen OpenAI API keys. This is why I started building a proxy for ourselves that allows for the creation of API keys with rate limits and cost controls. I built BricksLLM in Go since that was the language I used to build performative ads exchanges that scaled to thousands of requests per second at my previous job. A lot of developer tools in LLM ops are built with Python which I believe might be suboptimal in terms of performance and compute resource efficiency. One of the challenges building this platform is to get accurate token counts for different OpenAI and Anthropic models. LLM providers are not exactly transparent with the way how they count prompt and completion tokens. In addition to user input, OpenAI and Anthropic pad prompt inputs with additional instructions or phrases that contribute to the final token counts. For example, Anthropic's actual completion token consumption is consistently 4 more than the token count of the completion output. The latency of the gateway hovers around 50ms. Half of the latency comes from the tokenizer. If I start utilizing Go routines, might be able to lower the latency of the gateway to 30ms. BricksLLM is not an observability platform, but we do provide integration with Datadog so you can get more insights regarding what is going on inside the proxy. Compared to other tools in the LLMOps space, I believe that BricksLLM has the most comprehensive features when it comes to access control. Let me know what you guys think. https://ift.tt/aMu71sK December 12, 2023 at 12:26AM

Show HN: Slow Marathon https://ift.tt/p3vOugS

Show HN: Slow Marathon https://ift.tt/t8DTzOV December 11, 2023 at 02:26AM

Show HN: MyYogaFlow – Online Yoga Instructor https://ift.tt/CoKrcAh

Show HN: MyYogaFlow – Online Yoga Instructor Hi fellow hackers :) I am an indie hacker based in Switzerland who has been tinkering around with many side projects. Most of the things I developed were for myself or just to get familiar with a specific technology or piece of software. I never really deployed anything so far. Not too long ago, my SO and me bought a deck of cards with different yoga poses on them. Since we both work from our office chairs more than 8 hours a day, we wanted to do something for our body and this is where the yoga idea originated from. After the first 10 minutes of reading through the leaflet and another 30 minutes of choosing the poses and putting them in a somewhat meaningful order, we finally started with our own yoga session. It was clear to me: this is not something my SO wants to do every day. Doing the same order of poses every day wasn't an appealing alternative either. Away with the cards and on the lookout for an alternative again. Hence I started developing an application for us, that automatically generates a yoga session. It shows the poses one by one and guides you through the session. Each session is unique, which makes it more interesting over the days. The app even lets you create your custom flow that you select or it chooses among poses that fit your experience level just right. I have recently launched this application and made it publicly available. It is working (in most of the cases) and we are pretty happy about how it turned out. We used it many times in our lunch breaks or after a stressful day to calm down. I would be more than happy to get some feedback on it and some pointers on what to improve or focus on next. https://ift.tt/Wx7ATgS December 10, 2023 at 08:32PM

Show HN: Fine-grained stylistic control of LLMs using model arithmetic https://ift.tt/DPyVids

Show HN: Fine-grained stylistic control of LLMs using model arithmetic We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: https://ift.tt/xS3QOXB . Feedback and potential applications are welcome. https://ift.tt/Ku53ETQ December 10, 2023 at 02:27AM

Show HN: Seamless – An AI assistant that writes your literature review https://ift.tt/YMxak9T

Show HN: Seamless – An AI assistant that writes your literature review https://seaml.es/ December 10, 2023 at 01:09AM

Show HN: wallstreetlocal – View investments from the world's biggest SEC filers https://ift.tt/KvxujOW

Show HN: wallstreetlocal – View investments from the world's biggest SEC filers Hello Hacker News! My name is Anonyo, and I am a seventeen-year-old from Southeast Michigan. This is wallstreetlocal, my passion project for the last year. The United States Securities and Exchange Commission (SEC), forces institutional money managers whose portfolios surpass $100 million to quarterly update the SEC website with their latest investments. These money managers, known as 13F (the form type of the report) filers, are the biggest investors in America, and the law forces them to publicly reveal their investments. The problem I found was that SEC filings are cumbersome to find and view. To remedy this, wallstreetlocal saves and formats SEC data by bulk in an easily accessible format. Other websites do the same thing wallstreetlocal does, but wallstreetlocal places an emphasis on stock data. You can compare prices over time, view gains, and even download the data yourself to make any conclusions you wish. For as many stocks as possible, wallstreetlocal provides accurate, recent data to accompany the filings. As of current, there are 850,000 companies in the search database, 30,000 stocks in the stock database, and about 50 filers queried (equates to a couple million lines of JSON, requested by myself alone). By exploring the website, you can see the resources I used, check out some notable money managers I listed, and download any data that suits you. All for free. I plan to open-source this project someday, but I just wanted to get it released for now. The stack uses FastAPI for the back-end, and Next.js for the front-end. The back-end is hosted on a VM using Docker for the different microservices, and the front-end on Vercel. I made this project to better democratize SEC filings, and also to get some experience on my hands. I love computers, and one day hope to change the world through startups. In the comments, I'd appreciate any and all advice, as well as feedback on how to improve the site. Thanks. https://ift.tt/BNUHIXC December 9, 2023 at 11:26PM

Show HN: Open sourcing Restly – GPT-4 powered tutorials from your OpenAPI spec https://ift.tt/25okIxw

Show HN: Open sourcing Restly – GPT-4 powered tutorials from your OpenAPI spec Hi HN - Marius (mbuleandra) and I built Restly as a side project. Restly generates a user-friendly tutorial against any OpenAPI spec. While building public-facing APIs, we found that writing these guides by hand took a lot of work. Many OpenAPI specs were too large to send to GPT-4 / Claude directly, and even when we could send them, we experienced lost-in-the-middle and hallucination issues. The project is very alpha but works against the Vocode API when we specify well-defined goals. Let us know your thoughts; we would love your contribution to the project! [1] https://getrestly.com/ [2] https://ift.tt/FKzNqIU https://ift.tt/FKzNqIU December 9, 2023 at 04:04AM