1102
Daily Python News Question, Tips and Tricks, Best Practices on Python Programming Language Find more reddit channels over at @r_channels
Monday Daily Thread: Project ideas!
# Weekly Thread: Project Ideas 💡
Welcome to our weekly Project Ideas thread! Whether you're a newbie looking for a first project or an expert seeking a new challenge, this is the place for you.
## How it Works:
1. **Suggest a Project**: Comment your project idea—be it beginner-friendly or advanced.
2. **Build & Share**: If you complete a project, reply to the original comment, share your experience, and attach your source code.
3. **Explore**: Looking for ideas? Check out Al Sweigart's ["The Big Book of Small Python Projects"](https://www.amazon.com/Big-Book-Small-Python-Programming/dp/1718501242) for inspiration.
## Guidelines:
* Clearly state the difficulty level.
* Provide a brief description and, if possible, outline the tech stack.
* Feel free to link to tutorials or resources that might help.
# Example Submissions:
## Project Idea: Chatbot
**Difficulty**: Intermediate
**Tech Stack**: Python, NLP, Flask/FastAPI/Litestar
**Description**: Create a chatbot that can answer FAQs for a website.
**Resources**: [Building a Chatbot with Python](https://www.youtube.com/watch?v=a37BL0stIuM)
# Project Idea: Weather Dashboard
**Difficulty**: Beginner
**Tech Stack**: HTML, CSS, JavaScript, API
**Description**: Build a dashboard that displays real-time weather information using a weather API.
**Resources**: [Weather API Tutorial](https://www.youtube.com/watch?v=9P5MY_2i7K8)
## Project Idea: File Organizer
**Difficulty**: Beginner
**Tech Stack**: Python, File I/O
**Description**: Create a script that organizes files in a directory into sub-folders based on file type.
**Resources**: [Automate the Boring Stuff: Organizing Files](https://automatetheboringstuff.com/2e/chapter9/)
Let's help each other grow. Happy
/r/Python
https://redd.it/1g8cr07
Please help me create the perfect structure for my Flask App
I am making an URL shortener with Flask and MongoDB and just wanted to figure out the perfect project structure which follows the best practices.... I have already created this before but the structure sucks... I dont want auth right now and want to keep things modular... This is the current structure of the code:
C:.
│ .env.example
│ .gitattributes
│ .gitignore
│ botuseragents.txt
│ CODEOFCONDUCT.md
│ contributing.md
│ docker-compose.yml
│ dockerfile
│ emojies.py
│ LICENSE
│ main.py
│ pyproject.toml
│ railway.json
│ README.md
│ render.yaml
│ requirements.txt
│ vercel.json
│
├───.github
│ └───workflows
│
/r/flask
https://redd.it/1g7zb9h
D Simple Questions Thread
Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
Thanks to everyone for answering questions in the previous thread!
/r/MachineLearning
https://redd.it/1g80nkv
Mutmut 3 released
I've just released version 3 of mutmut, the industry standard mutation tester for Python. It's an almost complete rewrite to using mutation schemata, which (when coupled with some other changes) massively improve performance.
There is also a new terminal based UI for working with mutants that makes doing MT much faster and more fun.
Docs: https://mutmut.readthedocs.io/
Code: https://github.com/boxed/mutmut
/r/Python
https://redd.it/1g806h6
Are all the scientific python subreddits dead?
I have checked r/scipy and it doesn't look like it has had any posts for years. Where do people go to discuss scientific applications of python now? I have implemented a Biot Savart equation simulation I am looking for some feedback on.
/r/Python
https://redd.it/1g7t726
Unexpected Behavior with Flasgger
Hi all,
While I am versed in Python/Flask ...
I have a bit of an issue with my swagger docs - first timer here. I am open both to solutions using my current libraries and completely novel solutions where I tear it all down.
My current `YAML` implementation renders just fine on https://editor.swagger.io/ and passes other online `YAML` validation tools, so this is definitely a "well it works on X machine" sort of thing - I get it. Please help me my fellow Pythonistas!
While the most important file, my YAML is the longest so I pasted it at the end. Scroll if you want to skip. Not sure if it is this file or a problem with the flasgger lib.
relevant libraries:
flasgger 0.9.7.1
Flask 3.0.3
Flask-Cors 5.0.0
Flask-RESTful 0.3.10
# root/api/config.py
#!/opt/homebrew/bin python3.12
import os
# api config base class
/r/flask
https://redd.it/1g6079j
Firebase for authentication ?
Hey everyone! What do you think about using Firebase authentication in Django instead of Django Alluth?
Which one has been the better experience for you? ( or any other other auth provider like Supbase) , Thanks
/r/django
https://redd.it/1g7qfy2
Sunday Daily Thread: What's everyone working on this week?
# Weekly Thread: What's Everyone Working On This Week? 🛠️
Hello /r/Python! It's time to share what you've been working on! Whether it's a work-in-progress, a completed masterpiece, or just a rough idea, let us know what you're up to!
## How it Works:
1. Show & Tell: Share your current projects, completed works, or future ideas.
2. Discuss: Get feedback, find collaborators, or just chat about your project.
3. Inspire: Your project might inspire someone else, just as you might get inspired here.
## Guidelines:
Feel free to include as many details as you'd like. Code snippets, screenshots, and links are all welcome.
Whether it's your job, your hobby, or your passion project, all Python-related work is welcome here.
## Example Shares:
1. Machine Learning Model: Working on a ML model to predict stock prices. Just cracked a 90% accuracy rate!
2. Web Scraping: Built a script to scrape and analyze news articles. It's helped me understand media bias better.
3. Automation: Automated my home lighting with Python and Raspberry Pi. My life has never been easier!
Let's build and grow together! Share your journey and learn from others. Happy coding! 🌟
/r/Python
https://redd.it/1g7m7nw
Real-time YouTube Comment Sentiment Analysis with Kafka, Spark, Docker, and Streamlit
Hey r/Python! 👋
# What My Project Does:
This project performs real-time sentiment analysis on YouTube comments using a stack of Kafka, Spark, Docker, and Streamlit. It classifies comments into positive, neutral, or negative sentiments and displays the results in a web interface for easy visualization and interpretation. The aim is to provide insights on how users are reacting to YouTube videos in real-time, which can be especially useful for content creators, marketers, or analysts who want to track audience reception.
# Target Audience:
This project is primarily a learning-focused, proof-of-concept to demonstrate the power of real-time big data analytics with modern tools. While it could potentially be expanded into a production-ready system, it’s currently a toy project meant for educational purposes and exploring various technologies. Developers looking to explore Kafka, Spark, and Streamlit in a Dockerized environment will find this project helpful.
# Comparison:
What sets this project apart from existing alternatives is its real-time processing capability combined with the use of big data tools. Most sentiment analysis projects process data in batch mode or on a smaller scale, while this project uses Kafka for real-time streaming and Spark for distributed processing. It’s also containerized with Docker, which makes it easy to deploy and scale.
/r/Python
https://redd.it/1g7cs5l
FriendlyDateParser, just another Python module for date parsing!
Hello!
I've just released **FriendlyDateParser**, just another Python module for date parsing.
## What My Project Does
It can parse complex date expressions like:
* `2 days before the last day of next month`
* `1h15m after next Sunday at midnight CEST`
* `the second Monday of 2012`
The goal is to make working with date references straightforward, even when the expressions are complex.
## Target Audience
At this point, the module is still young and bugs should be expected. There may still be some edge cases which are not handled correctly.
## Comparison
Well, actually the reason for writing this module is that I had been using [**dateparser**](https://dateparser.readthedocs.io/) (*the* mature module with a similar purpose) for a while, but I found it was not able to handle all the cases I needed. So, I created **FriendlyDateParser** to address those gaps.
On the other hand, `dateparser` is multilingual while `friendlydateparser` only support English expressions (and I don't plan to extend the module in that way).
## Links
Check out the documentation and repo [here](https://github.com/salva/py-friendlydateparser)!
The module is also available from [PyPI](https://pypi.org/project/friendlydateparser/)
I'd love to hear your feedback and see how it works for you!
/r/Python
https://redd.it/1g6nwca
filefrag - library and executable to explore file fragmentation
Spent last night making this, added some turd polish today and added it to pypi.
## 🤷 why/what?
I wanted to get file fragmentation info so I can punch holes in files, aligned with memory pages. But I really didn't want to parse filefrag's outputs, so I wrote a python version with a friendly API and a command line that can produce json.
It only works on Linux as it depends on the FIE interface, but pull requests welcome etc.
## ⚒️ how?
See the video for a demo including installing from source, but you can install with pip:pip install filefrag
Then you can run pyfilefrag, see --help for details. It has --verbose, and --json outputs for your parsing pleasure.
To use the library, just call filefrag.FileMap('/path/whatever') to build a map of the extents in the file using ioctl's interface. Then you can poke about in the guts of a file:
⛓️💥 inspect fragmentation
🔍 find out where data is on your physical drive
🟰 compare extents between paths
📔 use them as dict keys
🕳️ check files for holes, like before and after hole punching
✅verify your XFS deduplication strategy, write your own stats tool
💩 dump file layouts to json (`print(f"{filemap:j}"`)
⚠️ break your disk
/r/Python
https://redd.it/1g70lqb
Best resources to learn
Having always use node js for my backends, I’m trying to widen my skills. What are the go to resources to learn Django?
/r/djangolearning
https://redd.it/1g6ull9
stick or move to another
So i made an inventory management system in Django for my internship project.
i chose Django for several reasons:
- i wanted to learn python and discover how to do weB apps with it
- i wanted to be unique because all of my colleagues used PHP (Laravel)
- Django has pretty much everything compared to Flask
- ...
Django is the first backend web framework i ever tried.
About the experience..., it was not that good i don't why.
Is it bc the result code IT WAS LITERALLY A MESS
i could not fix bugs or add features without breaking something
and then rollback to the previous version.
and sometimes i wanted to something simple but Django make it hard
like custom forms and how to integrate Django with CSS frameworks make it
an absolute MESS.
what i am missing in my knowledge to have a great experience with Django?
do i miss something in Django or in web/dev itself?
did you have some issues too with Django at your early days learning it?
should i move to something else like Laravel or Spring (ik some Java and PHP)
or just stick with Django?
bc i feel like it is not for me.
project repo
i hope i explained the problem.
thank you in advance.
/r/django
https://redd.it/1g6q4h5
Saturday Daily Thread: Resource Request and Sharing! Daily Thread
# Weekly Thread: Resource Request and Sharing 📚
Stumbled upon a useful Python resource? Or are you looking for a guide on a specific topic? Welcome to the Resource Request and Sharing thread!
## How it Works:
1. Request: Can't find a resource on a particular topic? Ask here!
2. Share: Found something useful? Share it with the community.
3. Review: Give or get opinions on Python resources you've used.
## Guidelines:
Please include the type of resource (e.g., book, video, article) and the topic.
Always be respectful when reviewing someone else's shared resource.
## Example Shares:
1. Book: "Fluent Python" \- Great for understanding Pythonic idioms.
2. Video: Python Data Structures \- Excellent overview of Python's built-in data structures.
3. Article: Understanding Python Decorators \- A deep dive into decorators.
## Example Requests:
1. Looking for: Video tutorials on web scraping with Python.
2. Need: Book recommendations for Python machine learning.
Share the knowledge, enrich the community. Happy learning! 🌟
/r/Python
https://redd.it/1g6wbs0
PyQt best option for commercial use?
I'm looking to possibly develop an app that will run on a Linux Desktop, specifically Ubuntu, and the latest OS X. The UI and performance are very important. Is PyQt my best option?
/r/Python
https://redd.it/1g6brra
Django's TechTuber?
There are Youtubers or content creators who are a reference about Django, to learn about the framework, to know the latest news?.
/r/django
https://redd.it/1g8f4xs
R Google Shopping 10M dataset for large scale multimodal product retrieval and ranking
We have finally released the Marqo Google Shopping 10 million dataset on Hugging Face (Marqo-GS-10M). One of the largest and richest datasets for multimodal product retrieval!
+ 10M rows of query, product title, image and rank (1-100)
+ \~100k unique queries
+ \~5M unique products across fashion and home
+ Reflects real-world data and use cases and serves as a good benchmark for method development
+ Proper data splits, in-domain, novel query, novel document and novel-document and novel query.
The dataset features detailed relevance scores for each query-document pair to facilitate future research and evaluation.
!pip install datasets
from datasets import loaddataset
ds = loaddataset("Marqo/marqo-GS-10M")
We curated this large-scale dataset as part of the publication of our training framework: Generalized Contrastive Learning (GCL).
Dataset: https://huggingface.co/datasets/Marqo/marqo-GS-10M
GCL: https://github.com/marqo-ai/GCL
Paper: https://arxiv.org/abs/2404.08535
/r/MachineLearning
https://redd.it/1g8a3pv
Digital ocean spaces for static file storage
I'm hosting a Django app on heroku and I want to use digital ocean spaces for serving static files. Does anyone have any insight on this? I'm pretty new to dev but really love how far I've come thanks to Django.
/r/django
https://redd.it/1g80s1k
Problem uploading files larger than 125kB
Hi guys!
There's a problem with a Django website, that I'm working on. The website has been setup a year ago at a hosting provider that uses cPanel and it has worked perfectly, but a few days ago something broke and now no one is able to upload any file larger than 125kB or groups of files, whose size is bigger than 125kB through the website. When someone tries to upload files larger than that, the page just simply loads till infinity without actually making any progress or creating the file on the disc. It's also important to not that when I tried to upload a file larger than 125kB in the Django admin panel and I left some fields empty, which can't be null, the infinite loading still happened, so it seems, that the problem is caused before the form validation takes place. These problems also aren't present when I try to upload the files on the local debug version.
We have one more Django based website at this provider and this happened to that as well. I even tried setting up a fresh Django project to test whether something broke, but the same issue arose. Meanwhile it's perfectly possible to
/r/django
https://redd.it/1g7utua
Deployed flask app with SQLite database doesn't have data persistance
I developed a flask application with a SQLite database and deployed it on Render however I realised that the being stored or updated to the application after deployment gets lost after some time. Upon further research online, I read that I should switch to a PostgreSQL database. However, I have the following questions:
1. Why is the deployed application failing to save data yet during development it worked fine?
2. If both SQLite and PostgreSQL databases are relational databases, how come PostgreSQL can maintain data persistence after deployment and SQLite databases can't?
3. What alternative deployment services can I use that support PostgreSQL databases and data persistence after deployment?
/r/flask
https://redd.it/1g5nfs7
Getting started with DRF for my VueJS frontend
Hello, so I posted something similar to this issue a week ago or less. But I'm trying to connect my Django project to my VueJS frontend pages that I created using DRF.
To my understanding I have to create API with DRF, so I was following the DRF docs tutorial and followed everything it said for coding the views, serializers, etc.... But then I ran `python manage.py runserver` and this caused all kinds of errors.
So I decided it was for the best to clone my project from my remote repo on Github and start all over with DRF. Now that I'm back to square one, my question is, what approach should I do to get started with DRF to create API end points for my frontend and backend? I want to try again with the DRF docs, but I don't want to fuck up my project again with an endless cycle of errors.
I'm beginning to think that I should just create a new app within my project, maybe that's what threw the errors I was getting before. What do you guys think? Here is the link to my Github repo, the project name is MyProject https://github.com/remoteconn-7891/MyProject.
/r/django
https://redd.it/1g7prk8
Feel like an ostracized freak for using Django as a monolithic websocket backend
Never did web-development before this and wanted to create the simplest possible websocket web app in python with a database.
I have Django serving my react.js frontend and handling the async websockets through channels. I don't use redis, celery, or microservices.
My backend code is async, so using one server process for multiple users works perfectly without scaling issues. I don't need a distributed task queue system since my async code is basically just waiting for IO. When I have something CPU intensive like gunzipping I just use a ProcessPoolExecutor to not block the event loop.
There's basically no documentation on how to set up the app the way I did and it's basically just been me hacking and experimenting with what works. I feel like I've been using django the wrong way since day one since every guide is either simple synchronous forum website, or straight to redis/celery/rabbitmq/kubernetes/gunicorn. There's no common ground.
/r/django
https://redd.it/1g7e8yf
Symlink in static dir to serve files
I understand that only images within the static dir can be displayed directly for security purposes.
Does creating a symlink within static, pointing to another folder, pose a security risk? How would someone go about exploiting this to access other data on the server?
/r/flask
https://redd.it/1g6n3ym
Creating simple inventory management app
Hi all, I'm trying to learn about Flask and decided to create a simple inventory management app which allows me to add, remove and edit entries in sqlite db using frontend. I was able to make the python app work but I'm stuck on the frontend part. I have the html file in "templates" folder and the the js script in the "static" folder, but when I try to run it, I end up with this error: "Failed to load resource: the server responded with a status of 404 (NOT FOUND)". Can someone help me out on what I'm missing?
Here's my repo: https://github.com/iraklikeshelava/inventory-management
/r/flask
https://redd.it/1g6fcyb
How to write the Terms and privacy section of the website ?
i got to that point where you must write the terms and privacy, and i was thinking if there is a custom terms or privacy to include or rules to follow when writing them, any suggestions ?
/r/djangolearning
https://redd.it/1g765b5
Django SQL Lite DB Built-in Size !?
I want to create a django app but for now i use a script on my PC the fetch some data and all the final results are stored into a json that can reach size of 600 to 700 MBs.... Can the DB store this size ? I have a hosting provider where to put the app on and unfortunate, my current plan doesn't have access to bigger DBs, only some that can store up to 350 MBs...
/r/django
https://redd.it/1g7022w
Aid with using anaconda to load fashionmnist
Hello could someone please step out how from the anaconda.navigator to load fashionmnist (which is on a laptop that’s never run it and may be missing things needed)
/r/Python
https://redd.it/1g6wjqj
My first full stack app
https://redd.it/1g6uq6p
@pythondaily
Lost my sqlite database😱. After that I wrote a backup script that runs once a day
This is just a warning to all you guys, remember to backup your database cause shit happens. Now I rest easy cause I wrote a backup script that runs automatically once a day just to backup my db, I know its going to fill up my space very soon and raise my costs but I do think its worth it, by the way the project I lost the db for was still a new project so there was not a lot of records in there anyway so not all hope is lost
/r/django
https://redd.it/1g6l5dv
I migrated from digital ocean to hetzner - (good experience)
I am building a streaming platform for my wife.
I tried to find the best option over all the cloud providers. At first I decided to use Digital Ocean because of their user friendly docs and UI. Also finding terraform docs was very easy. To be honest setting up the project was very easy so I can say I am satisfied with that.
However the outbound limit of 4TB, was a deal breaker for me. At the beginning I was thinking the limitting the video quality at 720P, however my wife wanted to have 1080P.
I started my search and find out that, hetzner was not only offering the cheap hardware but also the traffic limits were way better. To compare the numbers:
digital ocean: 24$
- 80gb ssd,
- 4gb ram
- 2Vcpu
- 4TB outbound
hetzner: 16€
- 160gb ssd
- 8gb ram
- 4Vcpu
- 20TB outbound.
So I decided to deploy my project on their servers. My concern was not finding good terraform docs for that, but tbh it was very easy and straight forward. In less then 3 hours I was able to run my
/r/django
https://redd.it/1g6kive