Today I spent time fixing issues that only show up once you start building real async systems.
The application looked like it was working, but the actual execution was not happening. The UI returned success, but the background pipeline was not running. This is one of the most dangerous states to be in because nothing crashes loudly.
The Root Problems
The root problems were all infrastructure related. The background worker was not running locally, so async jobs had nowhere to execute. Some environment variables still contained placeholder values, which caused AI calls to fail instantly. On top of that, incorrect model identifiers led to repeated 404 errors. While retrying and testing, the internal rate limiter and free tier API limits started blocking requests, which made debugging even harder.
Individually, these are small mistakes. Together, they create silent failure loops.
Validating Layer by Layer
The way out was to validate the system layer by layer instead of guessing. First confirming that background execution exists. Then checking external services independently. Then fixing model configuration and clearing cached error states so fresh requests could actually run.
Once this was done, the system moved from "request accepted" to "work actually executed".
The Main Learning Was Architectural
If you are building async, AI-heavy systems, you cannot rely on assumptions. You need visibility into background execution, failures, retries, and rate limits. Without that, you waste time debugging the wrong layer.
The product did not change today. The architecture did. And that matters more at this stage.