Show HN: SQLFrame – I ran PySpark without Spark on a SQL database https://ift.tt/BoqhELy

Show HN: SQLFrame – I ran PySpark without Spark on a SQL database Recently I open-sourced SQLFrame, a DataFrame library that implements the PySpark DataFrame API but removes Spark as a dependency. It does this by generating the corresponding SQL for the DataFrame operations using SQLGlot. Since the output is SQL this also means that the PySpark DataFrame API can now be used directly against other databases without the Spark middleman. I built this because of two common problems I have faced in my career: 1. I prefer to write complex pipelines in PySpark but they can be hard to read for SQL-proficient co-workers. Therefore I find myself in a tradeoff between maintainability and accessibility. 2. I really enjoy using the PySpark DataFrame API but not every project requires Spark and therefore I'm not able to use the DataFrame library I am most proficient in. The library currently focuses on transformation pipelines (reading from and writing to tables) and data analysis as key use cases. It does offer some ability to read from files directly but they must be small although this can be improved over time if there is demand for it. SQLFrame currently supports BigQuery, DuckDB, and Postgres with Clickhouse, Redshift, Snowflake, Spark, and Trino in development or planned. You can use the "Standalone" session to test running against any engine supported by SQLGlot but there could be issues with more advanced functions that will be resolved once officially supported by SQLFrame. Blog post for more details: https://ift.tt/FViUhDS... Would love to answer any questions or hear any feedback you may have! https://ift.tt/MZAPHjD May 21, 2024 at 06:39AM

Show HN: Recall Memory – Your Personal Local Time Machine for Mac https://ift.tt/AlQ2Dwf

Show HN: Recall Memory – Your Personal Local Time Machine for Mac Recall Memory is a free Sandboxed Mac App which let's you scroll through time and allows you to "Recall" what you were doing earlier. Recall works by capturing the active window every second and only saves screenshots with significant changes. Making it easy to scroll through what you were doing earlier. All data is processed and always stored locally. https://ift.tt/stoLZwx May 21, 2024 at 05:18AM

Show HN: Schedule SSH Execution of Bash/PowerShell/SQL Scripts https://ift.tt/a5YP8jd

Show HN: Schedule SSH Execution of Bash/PowerShell/SQL Scripts https://ctfreak.com May 21, 2024 at 02:04AM

Show HN: ffmpeg-english "capture from /dev/video0 every 1 second to jpg files" https://ift.tt/MiuvS3P

Show HN: ffmpeg-english "capture from /dev/video0 every 1 second to jpg files" https://ift.tt/g6nAJhj May 20, 2024 at 05:20AM

Show HN: Checkpoint 401 – forward auth server in TypeScript / Deno https://ift.tt/WN7SaOw

Show HN: Checkpoint 401 – forward auth server in TypeScript / Deno I wrote a forward auth server in TypeScript and Deno. I've written several forward auth servers before but they have always been specifically written for that application. I wanted something more generalised that I could re-use. What is forward auth? Web servers likes Nginx and Caddy and Traefik have a configuration option in which inbound requests are sent to another server before they are allowed. A 200 response from that server means the request is authorised, anything else results in the web server rejecting the request. This is a good thing because it means you can put all your auth code in one place, and that the auth code can focus purely on the job of authing inbound requests. Checkpoint 401 aims to be extremely simple - you define a route.json which contains 3 things, the method, the URL pattern to match against and the filename of a TypeScript function to execute against that request. Checkpoint 401 requires that your URL pattern comply with the URL pattern API here: https://ift.tt/SbfI8lL... Your TypeScript function must return a boolean to pass/fail the auth request. That's all there is to it. It is brand new and completely untested so it's really only for skilled TypeScript developers at the moment - and I suggest that if you're going to use it then first read through the code and satisify yourself that it is good - it's only 500 lines: https://ift.tt/T2s6d8K... https://ift.tt/9sKDAxe May 20, 2024 at 12:11AM

Show HN: A VS Code extension to check incompatible CSS https://ift.tt/aMOPi7L

Show HN: A VS Code extension to check incompatible CSS I've developed a handy tool for Visual Studio Code that makes checking CSS compatibility very easy for developers. It provides instant feedback on syntax, keywords, types, and function compatibility. It even highlights deprecated, non-standard, and experimental features. https://ift.tt/U1DPxS0 May 19, 2024 at 11:24PM

Show HN: I built an app that writes your life story https://ift.tt/9aJ8fAc

Show HN: I built an app that writes your life story https://memoir-app.com May 19, 2024 at 11:07PM