Overview
Soul-manga is an innovative single-page application developed using ReactJS, Python/Flask, SQLite, and Scrapy. This application offers an engaging platform for manga enthusiasts to access and read comics. The setup integrates various technologies to ensure a smooth user experience, while also handling backend processes efficiently. The application has been designed to cater specifically to web users, though it does not support mobile platforms.
Features
- Comprehensive Web Scraping: Utilizes Scrapy with three distinct methods to fetch manga data based on the URL type, ensuring comprehensive coverage of available content.
- Incremental Updates: The is_update parameter allows for efficient data fetching by only targeting recently updated pages, making the process quick and resource-efficient.
- Automated Deployment: Incorporates Gunicorn and Fabric as management tools to facilitate seamless deployment and server management.
- Scheduled Crawling: Using crontab to schedule crawls every 12 hours maximizes content freshness without overwhelming resources.
- Customizable Logging: Users can adjust logging levels and file settings in the configuration, ensuring that monitoring aligns with personal or project needs.
- Local Development Setup: Clear instructions for setting up the environment with Node.js and Python dependencies make it accessible for developers looking to contribute or modify the project.
