LocalAI Setup for Askimo App

LocalAI Configuration

Connect Askimo App to your self-hosted LocalAI instance for complete control over your AI infrastructure.

Server Configuration

• Server URL: LocalAI server endpoint
  • Default: http://localhost:8080
  • For remote servers: http://your-server:8080
• API Key: (Optional) If your LocalAI instance requires authentication
• Timeout: Connection timeout (default: 120s)
• Available Models: Detected from your LocalAI deployment

Setting Up LocalAI

1. Install LocalAI following the official guide
2. Start your LocalAI server:

docker run -p 8080:8080 --name local-ai -ti localai/localai:latest

3. Deploy models to your LocalAI instance
4. In Askimo, enter your LocalAI server URL
5. Click “Test Connection” to verify

Accessing Provider Settings

1. Click on the menu bar
2. Select “Settings”
3. Navigate to the “AI Providers” tab
4. Select “LocalAI” from the provider list

Keyboard Shortcut: ⌘ + , (macOS) or Ctrl + , (Windows/Linux) then click “AI Providers”

Deployment Options

Docker (Recommended):

docker run -p 8080:8080 \
  -v $PWD/models:/models \
  localai/localai:latest

Docker Compose:

version: '3.8'
services:
  localai:
    image: localai/localai:latest
    ports:
      - "8080:8080"
    volumes:
      - ./models:/models
    environment:
      - THREADS=4

Best Use Cases for LocalAI

Enterprise Deployments:

Process sensitive customer data with our internal AI infrastructure

Compliance Requirements:

Ensure all AI processing stays within our controlled environment

Custom Models:

Use our proprietary fine-tuned models for domain-specific tasks

Model Management

Installing Models: LocalAI supports various model formats:

• GGUF/GGML models
• PyTorch models
• ONNX models
• Custom backends

Model Gallery: Install models from the LocalAI gallery:

curl http://localhost:8080/models/apply \
  -H "Content-Type: application/json" \
  -d '{
    "id": "model-gallery://llama-2-7b-chat"
  }'

Troubleshooting

Cannot Connect?

• Verify LocalAI server is running
• Check firewall and network settings
• Ensure correct server URL and port
• Test with curl http://localhost:8080/v1/models

Authentication Issues?

• If your LocalAI requires authentication, add API key in settings
• Check LocalAI logs for auth errors
• Verify API key format matches LocalAI configuration

Model Loading Failures?

• Check LocalAI logs: docker logs local-ai
• Verify model files are in correct directory
• Ensure sufficient memory for model
• Try restarting LocalAI service

Slow Performance?

• Enable GPU acceleration
• Use quantized models
• Adjust THREADS environment variable
• Consider horizontal scaling with multiple instances

Advanced Configuration

Authentication: Configure API key authentication in LocalAI:

docker run -p 8080:8080 \
  -e API_KEY=your-secret-key \
  localai/localai:latest

Then add the API key in Askimo settings.

Custom Backend: LocalAI supports custom backends for specialized models:

name: custom-model
backend: custom-backend
parameters:
  model: /path/to/model

Resource Limits: Control resource usage:

docker run -p 8080:8080 \
  --memory="8g" \
  --cpus="4" \
  localai/localai:latest