Show HN: Crul – Query Any Webpage or API https://ift.tt/75SVopL

Show HN: Crul – Query Any Webpage or API Hi HN, we’re Carl and Nic, the creators of crul ( https://www.crul.com ), and we’ve been hard at work for the last year and a half building our dream of turning the web into a dataset. In a nutshell crul is a tool for querying and building web and api data feeds from anywhere to anywhere. With crul you can crawl and transform web pages into csv tables, explore and dynamically query APIs, filter and organize data, and push data sets to third party data lakes and analytics tools. Here’s a demo video, we’ve been told Nic sounds like John Mayer (lol) ( https://ift.tt/VJ0tswT ) We’ve personally struggled wrangling data from the web using puppeteer/playwright/selenium, jq or cobbling together python scripts, client libraries, and schedulers to consume APIs. The reality is that shit is hard, doesn’t scale (classic blocking for-loop or async saturation), and comes with thorny maintenance/security issues. The tools we love to hate. Crul’s value prop is simple: Query any Webpage or API for free. At its core, crul is based on the foundational linked nature of Web/API content. It consists of a purpose built map/expand/reduce engine for hierarchical Web/API content (kind of like postman but with a membership to Gold's Gym) with a familiar parser expression grammar that naturally gets the job done (and layered caching to make it quick to fix when it doesn’t on the first try). There’s a boatload of other features like domain policies, scheduler, checkpoints, templates, REST API, Web UI, vault, OAuth for third parties and 20+ stores to send your data to. Our goal is to open source crul as time and resources permit. At the end of the day it’s just the two of us trying to figure things out as we go! We’re just getting started. Crul is one bad mother#^@%*& and the web is finally yours! Download crul for free as a Mac OS desktop application or as a Docker image ( https://www.crul.com ) and let us know if you love it or hate it. ( https://ift.tt/I5eSbdQ ) And come say hello to us on our slack channel - we’re a friendly bunch! ( https://ift.tt/NYUyaGP ) Nic and Carl ( https://ift.tt/skKMVHD ) https://www.crul.com/ February 28, 2023 at 09:42PM

Show HN: Drai – AI Art Engine https://ift.tt/WYK01uX

Show HN: Drai – AI Art Engine https://ift.tt/JOzADKh February 28, 2023 at 06:28PM

Show HN: An automated code review tool using GPT3 https://ift.tt/IsNChQz

Show HN: An automated code review tool using GPT3 https://ift.tt/y8gI3rB February 28, 2023 at 08:44AM

Show HN: Collection of 2k+ Startup Incubators and Accelerators (global) https://ift.tt/6dtzsuO

Show HN: Collection of 2k+ Startup Incubators and Accelerators (global) https://ift.tt/yvDrHkF February 27, 2023 at 09:43PM

Show HN: AI Email Summaries https://ift.tt/IBAQzNt

Show HN: AI Email Summaries Harness the magic of AI built directly into your inbox with Smart Summaries. Summaries save you precious time by using AI to intelligently capture the main points of an email in just a few sentences. https://ift.tt/mbuKr5H February 27, 2023 at 10:46PM

Show HN: Go Bindings for Roc Toolkit https://ift.tt/Ol0V9R3

Show HN: Go Bindings for Roc Toolkit https://ift.tt/bsc9yCL February 27, 2023 at 09:39PM

Show HN: DbDeclare – A Python declarative layer for your database https://ift.tt/Z0IYEQs

Show HN: DbDeclare – A Python declarative layer for your database Hi HN! I made and just published v0.0.1 of DbDeclare. I use Python a lot, and interact with Postgres a lot. I like using SQLAlchemy, and I love Alembic. Those wonderful tools primarily operate on tables, though, and I often find myself writing custom code to declare what databases, roles, schemas, privileges, etc. I want, and I have a hard time updating them reliably and in a repeatable fashion. That's where DbDeclare aims to help: declare what you want in your cluster (in addition to SQLAlchemy-defined tables and columns) in-code, alongside your tables. There is a lot this can't do yet (thus the v0.0.1), but I think there's a decent foundation here to build on and eventually have really nice features like autogenerating change statements between your in-code definition and what is actually in your database cluster (like Alembic). This is also my first attempt at building an open-source project, so I'm sure there are plenty of mistakes. Please feel free to provide feedback, I'd love to make it better. For what it's worth, I'm aware that you can do some of this at the infrastructure-as-code layer using a tool like Terraform/Pulumi. My personal preference is to have all this sit closer to my tables rather than my infrastructure, so here we are. Anyway, let me know what y'all think. Thanks! https://ift.tt/jT1qxZM February 27, 2023 at 09:34PM