My use of AI

How I use AI

I use it principally to search. My agent has the ability to search ArXiV and I use it to summarise papers of interest.

I also use it to generate code, but I never let it commit. I read every single line. When I do commit, I make sure I share the billing with the model used.

I never use it to write as myself or anyone else.

I never send the personal information or intellectual property of myself or others to a hosted service without explicit permission.

My setup

Hardware

I have a Dell OptiPlex something-or-other running under my desk. It has an RTX Ada 2000 Generation and runs partially on solar power.

Software

I use ollama to serve models from that OptiPlex and Open WebUI to interact with models through a chat application, with hermes as an agent.

A docker-compose for this is:

services:
 ollama:
   image: ollama/ollama:latest
   container_name: ollama
   restart: unless-stopped
   ports:
     - "127.0.0.1:11434:11434"
   volumes:
     - ollama_data:/root/.ollama
   deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
 open-webui:
   image: ghcr.io/open-webui/open-webui:main
   container_name: open-webui
   restart: unless-stopped
   ports:
     - "127.0.0.1:3000:8080"
   environment:
     - OLLAMA_BASE_URL=http://ollama:11434
   volumes:
     - open_webui_data:/app/data
   depends_on:
     - ollama
 hermes:
   image: nousresearch/hermes-agent:latest
   restart: unless-stopped
   command: gateway run
   ports:
     - "127.0.0.1:8642:8642"
     - "127.0.0.1:9119:9119"
   volumes:
     - ~/.hermes:/opt/data
   environment:
     - HERMES_DASHBOARD=1
     - HERMES_DASHBOARD_BASIC_AUTH_USERNAME=${HERMES_DASHBOARD_BASIC_AUTH_USERNAME}
     - HERMES_DASHBOARD_BASIC_AUTH_PASSWORD=${HERMES_DASHBOARD_BASIC_AUTH_PASSWORD}
     - HERMES_DASHBOARD_BASIC_AUTH_SECRET=${HERMES_DASHBOARD_BASIC_AUTH_SECRET}

volumes:
 ollama_data:
   driver: local
 open_webui_data:
   driver: local
 hermes_data:
   driver: local

For development purposes, I use omp.

Models

Flavour of the month is Gemma4:12b, affectionally called Twelvebie. It works very well for what I use it for.

Twelvebie fits just fine on my RTX Ada Generation with room to spare for a few Pytorch projects.