# Site Update 2025 From 89 articles to 487 (including >80 totally new ones)! Well November is almost over so I thought I'd add some highlights and information about what we've done this month. ## AI & LLM Deep Dive The MAIN goal of this month for me was to get myself updated on all the 'AI' hype and understand the area better. I've been a developer for almost 30 years so I like to do this on occasion. WELL I went a bit crazy. As a result I wrote almost 40 articles on LLMs, RAG, ONNX, vision AI etc... ### Semantic Intelligence Series (10 Parts) A comprehensive deep-dive into building semantic search and AI-powered features for ASP.NET applications: 1. [Introduction to Semantic Intelligence](/blog/semantidintelligence) - Nov 13 2. [Part 1: Setting Up the Foundation](/blog/semantidintelligence-part1) - Nov 13 3. [Part 2: Embeddings and Vector Storage](/blog/semantidintelligence-part2) - Nov 13 4. [Part 3: Building the Search Pipeline](/blog/semantidintelligence-part3) - Nov 13 5. [Part 4: ONNX Integration](/blog/semantidintelligence-part4) - Nov 13 6. [Part 5: Advanced Querying](/blog/semantidintelligence-part5) - Nov 14 7. [Part 6: Hybrid Search Strategies](/blog/semantidintelligence-part6) - Nov 14 8. [Part 7: Performance Optimization](/blog/senmanticintelligence-part7) - Nov 15 9. [Part 8: Production Deployment](/blog/semanticintelligence-part8) - Nov 16 10. [Part 9: Monitoring and Maintenance](/blog/semanticintelligence-part9) - Nov 18 11. [Part 10: Future Directions](/blog/semanticintelligence-part10) - Nov 19 ### Building a Lawyer GPT for Your Blog (8 Parts) Building a complete RAG-based chatbot that can answer questions about your blog content: 1. [Part 1: Architecture Overview](/blog/building-a-lawyer-gpt-for-your-blog-part1) - Nov 12 2. [Part 2: Data Ingestion](/blog/building-a-lawyer-gpt-for-your-blog-part2) - Nov 12 3. [Part 3: Embedding Generation](/blog/building-a-lawyer-gpt-for-your-blog-part3) - Nov 12 4. [Part 4: Vector Storage with Qdrant](/blog/building-a-lawyer-gpt-for-your-blog-part4) - Nov 12 5. [Part 5: Query Processing](/blog/building-a-lawyer-gpt-for-your-blog-part5) - Nov 12 6. [Part 6: Response Generation](/blog/building-a-lawyer-gpt-for-your-blog-part6) - Nov 12 7. [Part 7: UI Integration](/blog/building-a-lawyer-gpt-for-your-blog-part7) - Nov 12 8. [Part 8: Deployment and Scaling](/blog/building-a-lawyer-gpt-for-your-blog-part8) - Nov 12 9. [Cloud Alternative Approach](/blog/building-a-lawyer-gpt-cloud-alternative) - Nov 12 ### RAG Fundamentals Series A practical guide to Retrieval-Augmented Generation: 1. [RAG Primer: Understanding the Basics](/blog/rag-primer) - Nov 22 2. [RAG Architecture Deep Dive](/blog/rag-architecture) - Nov 22 3. [Practical RAG Applications](/blog/rag-practical-applications) - Nov 22 4. [Hybrid Search and Indexing Strategies](/blog/rag-hybrid-search-and-indexing) - Nov 22 ### Other AI/LLM Articles - [LLM API Integration](/blog/llmapi) - Nov 2 - Building APIs for LLM services - [LLM API with gRPC](/blog/llmapigrpc) - Nov 6 - High-performance LLM APIs using gRPC - [Chunking and Caching Strategies](/blog/chunkingandcaching) - Nov 6 - Optimizing document processing for LLMs - [Avalonia TinyLLM Chat](/blog/avalonia-tinyllm-chat) - Nov 10 - Cross-platform desktop LLM chat application - [Efficient Alt Text Generation](/blog/efficient-alt-text-generation) - Nov 20 - Using vision models for accessibility - [Self-Hosted Vector Databases with Qdrant](/blog/self-hosted-vector-databases-qdrant) - Nov 23 - Running your own vector DB - [Semantic Search with ONNX and Qdrant](/blog/semantic-search-with-onnx-and-qdrant) - Nov 25 - Local semantic search implementation - [Why Commercial AI Projects Are Dumb](/blog/whycommercialaiprojectsaredumb) - Nov 26 - A critical look at enterprise AI hype I also wrote over a dozen tools and NuGet packages (many still awaiting final release) you can see [over here](https://github.com/scottgal/mostlylucid.nugetpackages). Packages like [mostlylucid.llmalttext](https://github.com/scottgal/mostlylucid.nugetpackages?tab=readme-ov-file#mostlylucidllmalttext) are aimed at solving problems I've had - in this case [generating alt text for images using vision LLMs](/blog/mostlylucid-llmalttext) (Nov 25). ## ASP.NET Core Deep Dives ### ASP.NET Pipeline Series (6 Parts) Understanding the ASP.NET Core request pipeline from the ground up: 1. [API Contexts](/blog/API-CONTEXTS) - Nov 5 - Understanding HttpContext and friends 2. [Server and Hosting](/blog/ASPNET-PIPELINE-PART2-SERVER-HOSTING) - Nov 9 3. [Middleware Deep Dive](/blog/ASPNET-PIPELINE-PART3-MIDDLEWARE) - Nov 9 4. [Routing and Endpoints](/blog/ASPNET-PIPELINE-PART4-ROUTING-ENDPOINTS) - Nov 9 5. [Application Models](/blog/ASPNET-PIPELINE-PART5-APPLICATION-MODELS) - Nov 9 6. [Advanced Extensibility](/blog/ASPNET-PIPELINE-PART6-ADVANCED-EXTENSIBILITY) - Nov 9 7. [State Between Requests](/blog/ASPNET-STATE-BETWEEN-REQUESTS) - Nov 9 ### Background Services Series 1. [Background Services Part 1: Fundamentals](/blog/background-services-in-aspnetcore-part1) - Nov 27 2. [Background Services Part 2: Advanced Patterns](/blog/background-services-in-aspnetcore-part2) - Nov 27 ### Other ASP.NET Articles - [OpenAPI Features in .NET 9](/blog/oprnapi-features) - Nov 5 - [HTMX with ASP.NET Core Partials](/blog/htmx-aspnetcore-partials) - Nov 28 - [Fixing SEO for ASP.NET Core Blogs](/blog/fixing-seo-for-aspnetcore-blog) - Nov 26 - [EF Migrations The Right Way](/blog/efmigrationstherightway) - Nov 23 - [Modern CQRS and Event Sourcing](/blog/moderncqrsandeventsourcing) - Nov 13 - [Backpressure in Queueing Systems](/blog/backpressure-queueing-systems) - Nov 23 - [SignalR Demo Guide](/blog/signalr-demo-guide) - Nov 4 - [Secure Chat Demo](/blog/securechatdemo) - Nov 11 ## Testing & Quality - [HttpClient Testing Without Mocks](/blog/httpclient-testing-without-mocks) - Nov 29 - Real HTTP testing strategies - [PuppeteerSharp E2E Testing](/blog/puppeteersharp-e2e-testing) - Nov 27 - Browser automation testing - [Playwright E2E Testing](/blog/playwright-e2e-testing) - Nov 27 - Modern E2E testing framework - [Using CSX for Testing](/blog/using-csx-for-testing) - Nov 26 - C# scripting for test scenarios ## Docker & DevOps - [Docker Development Deep Dive](/blog/docker-development-deep-dive) - Nov 9 - [Docker Swarm Cluster Guide](/blog/docker-swarm-cluster-guide) - Nov 12 - [Modernising Frontend Build Pipeline](/blog/modernising-frontend-build-pipeline) - Nov 11 ## The Site As part of my AI exploration I also updated the site to use a RAG approach for searching the blog posts. This uses a vector database (Qdrant) and ONNX (local, CPU) embeddings to allow semantic search of the blog posts. It also allows for things like smart 404 handling (by looking at incoming requests and matching them to existing posts). Related articles: - [The War on 404](/blog/the-war-on-404) - Nov 23 - Smart 404 handling with semantic matching - [Filter Bar Progress and Issues](/blog/filterbarprogressandissues) - Nov 10 - [Filter Bar Progress and Bugs](/blog/filter-bar-progress-and-bugs) - Nov 10 ### Mermaid Diagram Enhancements I published an [npm package](https://www.npmjs.com/package/mostlylucid-mermaid) for enhanced Mermaid diagrams: - [Mermaid JS Deep Dive](/blog/mermaid-js-deep-dive) - Nov 9 - [Enhancing Mermaid Diagrams with PanZoom and Export](/blog/enhancingmermaiddiagramswithpanzoomandexport) - Nov 7 - [Publishing Mermaid Enhancements to NPM](/blog/publishingmermaidenhancementsnpm) - Nov 7 ### Other Site Updates - [Paging Tag Helper](/blog/pagingtaghelper1) - Nov 7 - Custom tag helper for pagination - [Markdig Fetch Extension](/blog/markdigfetchextension) - Nov 7 - Fetching external content in Markdown ## EasyNMT → mostlylucid-nmt So for a WHILE now I've used the excellent [EasyNMT](https://github.com/UKPLab/EasyNMT) to translate posts on this site, however the original project has bitrotted. It was funded by an EU grant as a research project so has now been neglected for a few years. SO I [made my own](https://github.com/scottgal/mostlyucid-nmt). The docker image `mostlylucid-nmt` is fully backwards compatible BUT adds a TON of functionality and reliability features like: - Multi-model fallback - automatically falls back to less performant models to get max translation between THOUSANDS of language pairs - Memory management - manages its own memory (evict, load etc) so it can run FOREVER reliably - Enhanced GPU build with a TON of config - ARM64 versions (it CAN run on a Pi) - And MANY more around reliability, sentence splitting, markdown handling etc Related articles: - [mostlylucid-nmt Complete Guide](/blog/mostlylucid-nmt-complete-guide) - Nov 8 - [How Neural Machine Translation Works](/blog/how-neural-machine-translation-works) - Nov 9 ## mostlylucid.co.uk Content Resurrection My ambition when I STARTED this blog last year was to get ALL of my old content from my (2004-2010) site mostlylucid.co.uk moved over here. This was an ISSUE as I'd lost the data over the years so I wrote an [Archive.Org](https://archive.org/) data extraction tool which got the old content, extracted it, assigned dates, categories etc to fit the new format and now my very first blog posts are now up [here](/blog/538). - [Recovering Old Blog Content](/blog/oldblog) - Nov 24 - The full story of content recovery - [Reanimation](/blog/reanimation) - Nov 24 - Bringing old content back to life This ALSO meant a ton of updates to the site to handle outgoing broken links; it auto creates Archive.org links for old posts contemporaneous with the original article. ## Using Claude Code One of the BIG productivity wins this month was using [Claude Code](https://docs.anthropic.com/en/docs/build-with-claude/code-assistance). It let me complete DOZENS of unfinished articles that had been sitting in drafts. The AI assistance for: - Filling in code examples - Expanding on technical concepts - Fixing bugs in sample code - Generating test scenarios - Writing comprehensive series with consistent structure ...has been invaluable. It's like having a tireless pair programmer who never gets annoyed at "just one more thing". ## Opinion & Essay Pieces A few non-technical pieces on software development culture: - [Agile Standups: Ceremony Tax](/blog/agile-standups-ceremony-tax) - Nov 21 - Why daily standups often waste time - [Make It Work, Then Make It Pretty](/blog/makeitworkthenmakeitpretty) - Nov 19 - On iterative development - [The Elevator Pitch](/blog/elevatorpitch) - Nov 19 - Communicating technical ideas - [Playing With Code: Efficient Agile](/blog/playing-with-code-efficient-agile) - Nov 23 - [Writing Feature Specs](/blog/writingfeaturespecs) - Nov 11 - [A Word on Licensing](/blog/awordonlicensing) - Nov 23 - [Hidden Systems](/blog/hiddensystems) - Nov 11 - The invisible infrastructure we depend on ### The "DISE" Series (Developer Industry Software Engineering) Some thoughts on the software industry: - [DISE: Just Voyager](/blog/disejustvoyager) - Nov 24 - [Part 2: Apprenticeships](/blog/blog-article-cooking-dise-part2-apprenticeships) - Nov 18 - [Part 3: Untrustworthy Gods](/blog/blog-article-cooking-dise-part3-untrustworthy-gods) - Nov 20 ## Fun Stuff - [Rampage](/blog/rampage) - Nov 8 - A trip down memory lane ## What's Next? I'll ease off a bit...I still have articles for all the not yet released nuget packages but I'm happy with my AI knowledge for now. December will likely see me: - Publishing more of the NuGet packages I've been working on - Continuing the semantic search improvements - More RAG experiments Thanks for reading-see you in December!