Melvin Vivas

AI Engineering Projects

1. 01

   AIBackends

   AI APIs and local multimodal pipelines

   • Built the AIBackends API server, a TypeScript/Hono server supporting local runtimes and cloud providers, with text processing, vision, structured OCR, and tool-using agents. Added multi-turn sessions, streaming responses, custom HTTP tools, and MCP server integration.
   • Built the AIBackends Python library around llama.cpp and Transformers, with typed extraction, classification, embeddings, PII redaction, moderation, and reusable batch workflows.
   • Built local personally identifiable information (PII) detection and redaction in AIBackends using GLiNER zero-shot entity extraction and a local privacy-filter model, keeping sensitive data off cloud APIs.
   • Implemented sales-call analysis combining Whisper transcription, PII redaction, and structured reports; and video-ad analysis combining sampled frames, audio transcription, vision extraction, and Pydantic validation.
2. 02

   Coworker

   Desktop agents and task automation

   • Built Coworker, a local-first desktop application for AI coworkers using local or cloud models, extensible skills, web search, persistent memory, and approval controls.
   • Implemented recurring and one-time scheduling, coworker handoffs, and Telegram/Discord integrations for conversations, files, and approvals.
3. 03

   Notebook software factory

   Automated Colab notebook pipeline

   • Built a small-scale software factory with Grok Bot and Cursor cloud agents: it finds new open models on X, codes a Colab notebook for each, tests it in Google Colab, and merges it automatically after verification.
   • Implemented 40+ small open-model notebooks, including Qwen3.5, Liquid AI LFM3, Liquid AI d1 decision models, Gemma 4, and NVIDIA Nemotron 3.5.
   • Fine-tuning: Unsloth QLoRA on Gemma 4 E2B (tweet style, task routing), Qwen3.5-4B and Gemma 4 E4B decision models for email triage, and a ModernBERT classifier.
4. 04

   Agent orchestration and observability

   • Created Codex Astra Luna Orchestrator, reusable profiles separating orchestration, exploration, implementation, testing, and independent review, with cross-platform installers.
   • Built Agent Monitor to visualise agent trees and inspect traces, tool calls, token usage, and estimated costs across Codex, Claude Code, Pi, and Hermes.
   • Built Pi Agent Council, a multi-model advisory extension that runs independent read-only consultations in parallel and reports agreements, disagreements, assumptions, and usage.
5. 05

   Business workflow automation

   • Built an Apache Camel email workflow that classifies email and drafts replies through parallel AI calls, recording results and partial failures in SQLite for review.
   • Created CamelFlow, a Java application that turns Camel YAML routes into interactive diagrams with file watching, step inspection, and image export.
6. 06

   Evaluation and local model infrastructure

   • Built Local Evals for document-to-JSON, text-to-JSON, and tool-call evaluation, with experiment grouping, run comparisons, logs, and CLI execution. Implemented deterministic checks for tool names, arguments, ordering, and parameter schemas.
   • Built Open Model Server, a Go service with an embedded dashboard, SQLite model registry, model lifecycle management, GGUF serving through llama.cpp, and in-process ONNX inference.
   • Built Jev Dev, a tool for Jev testing and experiments.
   • Fine-tuned a Qwen3.5 2B model on my own X posts, published as qwen3.5-2b-melvin-posts-v1-GGUF on Hugging Face.
7. 07

   Mobile agents and generative media

   • Built Sidekick for iOS, a prototype personal AI assistant for iPhone powered by a local Gemma 4 E2B model. A SwiftUI assistant with a streaming tool-calling loop, image/file inputs, document creation, image/video generation integrations, memory, and approval-gated calendar and reminder actions.
   • Created AI Dream Photo, an AI photo-generation application with face personalisation and payment functionality. Fine-tuned Flux image models using LoRA for personalised photo generation.
8. 08

   HomerGPT

   Applied AI in real estate

   • Developed HomerGPT at Ohmyhome, using OpenAI function calling to provide real-estate insights in Singapore.

Skills and Technologies

AI agents and integration

Tool/function calling, multi-turn sessions, multi-agent orchestration, MCP, Claude Agent SDK, LangChain, CrewAI, OpenAI, Anthropic, Gemini, OpenRouter.

Local models and applied ML

Ollama, LM Studio, llama.cpp, Hugging Face Transformers, GGUF, ONNX Runtime, embeddings, zero-shot classification and extraction, classifier fine-tuning demonstrations, fine-tuning on personal data, MiniLM embeddings. Familiar with decision models, including Liquid AI d1 decision models and Jev.

Multimodal AI

Vision/OCR, structured document extraction, Whisper transcription, audio/video analysis, image generation, Flux (LoRA fine-tuning), Pydantic and JSON Schema validation, video understanding with Gemini Video Understanding API and local Qwen models, local personally identifiable information (PII) detection and redaction with GLiNER.

Evaluation and observability

Structured-output and tool-call grading, experiment comparisons, agent tracing, token/cost analysis, MLflow tracing and evaluations.

Automation and delivery

Apache Camel, n8n, scheduled agents, REST APIs, microservices, AI adoption and advocacy, developer relations, Agile/Scrum, JIRA.

AI coding tools

Codex, Claude Code, Cursor, Pi Agent.

Languages and development

Python, TypeScript/Hono, Go, Java/Spring Boot, Swift/SwiftUI, Next.js, Jupyter/Colab, VS Code, PyCharm, GoLand.

Data and deployment

PostgreSQL, SQLite, Elasticsearch, Neo4j, Supabase, Docker, AWS, GCP, GitHub, Bitbucket, Vercel, Netlify, CI/CD. GPU infrastructure: RunPod, Hugging Face Inference Endpoints.