Posted inAI

Build Your Own Internal ‘GitHub Copilot’ with Tabby: Maximum Security, Ultra-Fast Speed

Artificial Intelligence tutorial - IT technology blog
Artificial Intelligence tutorial - IT technology blog

Concerns About Source Code Leaks with AI

When working on Enterprise projects, the hardest part is balancing productivity and security. Developers crave the power of a premium AI assistant to replace GitHub Copilot, but managers often hesitate, fearing sensitive source code might be pushed to the cloud. In fact, many large corporations have banned third-party AI after serious data breach incidents.

After testing several options, I chose Tabby as the solution. It’s an open-source AI assistant that allows you to self-host entirely on your own infrastructure, much like building your own OpenAI API server. All code suggestion logic stays within your internal server. My team’s experience shows impressive response speeds with almost zero latency when configured correctly.

Up and Running in 5 Minutes with Docker

To test it quickly, you can spin up a Tabby instance using your CPU. Ensure you have Docker installed and run the following command:

docker run -it \
  -p 8080:8080 \
  -v $HOME/.tabby:/data \
  tabbyml/tabby serve --model TabbyML/StarCoder-1B --device cpu

Wait a few minutes for the model to download. Once the terminal indicates success, access http://localhost:8080 to set up your admin account. That’s it—you now have a private AI server ready to serve.

Why is Tabby Better Than Its Competitors?

Many of you might ask: Why not use Ollama or other MCP servers? Here are three reasons why Tabby excels in daily coding tasks:

  • Designed for Teams: Tabby uses a standard Client-Server architecture. A powerful server can easily handle 20-30 developers simultaneously without bottlenecks.
  • Project Indexing: It doesn’t just provide generic suggestions. Tabby can scan your entire codebase to learn your coding style and internal libraries.
  • Professional Dashboard: You can manage tokens, monitor logs, and switch models directly through the web interface without touching complex configuration files.

Optimizing for Production (Using GPU)

Using the CPU is just for testing. For real-world work, you’ll need an NVIDIA GPU (at least an RTX 3060) to reduce latency to under 200ms. Waiting 2 seconds for a code suggestion is extremely frustrating.

Step 1: Enable NVIDIA Container Toolkit

Docker needs this toolkit to leverage the power of your graphics card. On Ubuntu, run the following commands:

# Add repository and quick install
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

Step 2: Run a “High-End” Model

Instead of the somewhat naive 1B version (1 billion parameters), I recommend using Deepseek-Coder-6.7B. This is a highly intelligent model capable of understanding complex logic, similar to the reasoning capabilities discussed in mastering DeepSeek-R1 on Linux:

docker run -it \
  --gpus all \
  -p 8080:8080 \
  -v $HOME/.tabby:/data \
  tabbyml/tabby serve --model TabbyML/DeepseekCoder-6.7B --device cuda

Connecting to VS Code and JetBrains

Now that the server is running, it’s time to bring it into your workflow. The process is straightforward:

  1. Go to Extensions, search for “Tabby”, and click Install.
  2. Press Ctrl+Shift+P and type “Tabby: Settings”.
  3. Enter your server URL (e.g., http://192.168.1.50:8080).
  4. Copy the API Token from the web Dashboard and paste it into your IDE for authentication.

Try typing a data processing function. You’ll see ghost text suggestions appearing. Just press Tab, and everything will complete automatically as if a real AI assistant is writing it for you.

The ‘Killer’ Feature: Automated Indexing

This is my favorite part. Tabby understands your folder structure and which functions are called from which files in your project. To enable this, open the config.toml file and declare your repository:

[repositories.my-backend]
git_url = "https://github.com/your-org/core-api.git"

Once indexing is complete, the AI will prioritize suggestions following the specific patterns defined by your team.

Pro-Tips to Avoid Headaches

After several months of deployment, I’ve gathered three important takeaways:

  • Network Latency: If working remotely via VPN, high latency will ruin the smooth experience. Prioritize placing the server as close to the workspace as possible.
  • VRAM is Critical: A 7B model requires about 14GB of VRAM. If your card only has 8GB, look for Quantized versions to run more stably.
  • Information Security: Never expose port 8080 to the public internet. Use Tailscale or Cloudflare Tunnel to create a secure private connection for your team.

Self-hosting Tabby isn’t just about security; it’s about taking full control of your productivity tools, including an AI agent for automated coding. Good luck with your setup! If you encounter any issues with GPU configuration, feel free to leave a comment, and I’ll help you out!

Share:
Tags: