- Created a new JSON file `_journal.json` to track changes and entries.
- Included metadata such as version, dialect, and a sample entry with breakpoints enabled.
- Updated `SupplierAnalysisResult` to include a `product` field and modified related tests.
- Refactored `addRowsSheet` to accommodate changes in the product structure.
- Enhanced UPC file analysis to utilize a new `toSupplierInputRecord` function for cleaner record creation.
- Introduced new types for supplier input records and product observations.
- Updated frontend components to handle new product details and analysis history.
- Improved database writing functions to streamline run completion and error handling.
- Added new API endpoints for product details and adjusted routing in the frontend.
- Implemented supplier scoring logic in `supplier-scoring.ts` with functions to compute demand score, competition penalty, and overall supplier product score.
- Created unit tests for supplier scoring in `supplier-scoring.test.ts` to validate scoring logic against various scenarios.
- Developed UPC file analysis tool in `upc-file-analysis.ts` to process UPCs in batches, fetch product data from Keepa and SP-API, and generate supplier results.
- Added UPC input reading functionality in `upc-file-reader.ts` to handle XLSX and XLS files, including validation for UPC formats.
- Introduced a command-line tool in `upc-lookup.ts` for looking up UPCs and displaying detailed results or mappings to ASINs.
- Enhanced error handling and logging throughout the new modules for better traceability and user feedback.
- Replaced direct SQLite database calls with Drizzle ORM methods in `top-monthly-sold-by-category.ts`, `writer.ts`, and `upc-file-analysis.ts`.
- Updated test cases in `top-monthly-sold-by-category.test.ts` to mock the new database interactions.
- Removed unnecessary database initialization and cleanup code.
- Improved code readability and maintainability by using ORM features for inserting and updating records.
- Introduced `useClaude` option in `AnalysisPipelineOptions` to toggle Claude LLM usage.
- Updated `processProductChunk` and `analyzeProducts` functions to accept and handle `useClaude` parameter.
- Modified argument parsing in various scripts (`bestsellers-by-category`, `mid-range-sellers-by-category`, `top-monthly-sold-by-category`, etc.) to include `--claude` flag.
- Enhanced `analyzeProductsInternal` to differentiate between LLM providers and handle requests to Claude API.
- Added error handling for Claude API responses and ensured proper configuration for using Claude.
- Updated documentation and usage messages to reflect the new `--claude` flag.
- Introduced a new script `asin-offer-search.ts` for searching product offers by ASIN.
- Updated `package.json` to include a new command for the ASIN offer search.
- Enhanced configuration in `config.ts` to support SearXNG URL and timeout settings.
- Added comprehensive tests for the new search functionality in `searxng.test.ts`.
- Implemented the core search logic in `searxng.ts`, supporting multiple providers and price detection.
- Introduced StalkerResultItem and StalkerResultsResponse types for handling API responses.
- Implemented StalkerExplorer component for displaying Stalker results with search and filter options.
- Added sorting functionality for Stalker results table.
- Enhanced Dashboard to include a button for navigating to Stalker results.
- Updated routing to support Stalker results page.
- Improved styles for section headers and inventory columns in the results table.
- Add `writeSupplierWorkbook` function to create Excel workbooks for supplier analysis results.
- Introduce `SupplierExportSummary` type for summarizing export data.
- Create tests for `writeSupplierWorkbook` to ensure correct sheet creation and data population.
- Implement supplier scoring logic in `supplier-scoring.ts` to evaluate product profitability and demand.
- Add tests for supplier scoring to validate scoring logic and verdict determination.
- Enhance UPC file analysis to integrate supplier scoring and export results to Excel.
- Update database writing logic to accommodate new supplier analysis results.
- Refactor types to include supplier-specific data structures and scoring metrics.
- Ensure proper cleanup of temporary files after tests.
- Updated test case to reflect changes in processing mid-range matches based on sellability.
- Modified `processCategory` function to implement strict and soft sellability gates.
- Introduced new command-line arguments for category selection and sellability gate configuration.
- Enhanced error handling and validation for new arguments.
- Improved logging for category processing and budget usage.
This new pipeline identifies products meeting specific monthly sold, price, seller count, and Amazon buy box share criteria across categories. It fetches comprehensive product data from Keepa and SP-API, analyzes it using an LLM, and persists the results.
A key enhancement is the introduction of a dedicated Redis cache for Keepa and SP-API responses. This reduces API token consumption and improves performance for subsequent runs by caching enriched ASIN data with a 12-hour TTL. Products are saved regardless of their sellability status to provide a complete view.
Reduces API token consumption by disabling stats and buybox data for UPC-to-ASIN mapping requests. Additionally, introduces a run-level cache to avoid redundant lookups for the same UPC across different batch chunks.
Introduces the capability to resolve UPCs to ASINs using the Keepa API. This includes a new `upc-file` command for processing large Excel files of UPCs, a `upc` CLI tool for quick lookups, and API endpoints for web-based integration. The analysis pipeline was refactored into a reusable module to support both standard ASIN leads and new UPC-driven workflows.
- Introduced `amazonIsSeller` and `amazonBuyboxSharePct90d` fields in KeepaData type.
- Updated database schema and queries to store Amazon seller status and buy box share percentage.
- Enhanced product analysis results with new metrics from Keepa API.
- Modified frontend components to display Amazon seller status and buy box share percentage.
- Implemented reanalysis functionality for products to refresh Amazon-related metrics.
- Introduces a new script to discover categories and identify products with high monthly sales volume.
- Filters candidates based on sellability status and a configurable monthly sold threshold.
- Enriches product data with Keepa and SP-API metrics before performing LLM-based analysis.
- Persists results to SQLite and includes unit tests for the core processing logic.
- Adds new columns to the `results` table for tracking spreadsheet-sourced financial data, supplier details, and lead metadata.
- Implements `ensureResultsTableColumns` to automatically migrate existing databases with the new schema.
- Simplifies LLM evaluation logic by removing mandatory skip triggers for Amazon-exclusive and single-seller products.
- Standardizes formatting for database index and table creation statements.
- Implemented main dashboard with run metrics and filtering options.
- Created detailed view for individual runs with results and anomalies.
- Added product listing page with filtering and pagination.
- Introduced utility functions for formatting dates and numbers.
- Styled components with CSS for a clean and responsive layout.
- Set up HTML entry point and linked to the main JavaScript file.
- Updated TypeScript configuration to include DOM types.
- Replaces Excel output with structured database tables for tracking category analysis runs and product results.
- Implements a blacklist to exclude specific category IDs from the bestseller pipeline.
- Adds unit tests for category processing and enhances logging with levels and timestamps.
- Introduces foreign key enforcement and updated schema definitions in the database module.
- Updated `sp-test.ts` to enhance argument parsing and error handling for sellability checks.
- Refactored `types.ts` to maintain consistent formatting and improve readability.
- Improved `writer.ts` for better result handling and CSV writing, ensuring clarity in output.
- Adjusted `tsconfig.json` formatting for consistency and readability.
