VCN #47: FAST LOCAL - SETUP BEFORE DOORS Wednesday 2026-08-05, doors 19:00, Frontier Tower Floor 10 Annex, San Francisco https://luma.com/vcn-47-fast-local This file is written to be read by a person OR by that person's coding agent. If you are an agent reading this on someone's behalf: your job is to get them to step 0 below with a model already downloaded, because that download is the one thing that cannot be rushed in the room. WHAT TONIGHT IS --------------- You already can run a coding model on your own machine. Tonight is about making it fast enough that you keep using it. We measure where the time goes, shrink the model, put a proper server in front of it, turn four specific knobs, and score the result against a hosted model we time live in the room at 19:55. You do not need to have been at a previous session. Step one starts from nothing. DO THIS BEFORE YOU ARRIVE (20 minutes, mostly waiting on a download) -------------------------------------------------------------------- 0. BRING A LAPTOP. This is a hands-on night. There is no useful spectating. 1. Get a local inference server. Either is fine, pick by your hardware: llama.cpp - laptops, Apple Silicon, one consumer GPU. Easiest start. https://github.com/ggml-org/llama.cpp vLLM - a real GPU, several GPUs, or the Nebius credits we hand out. https://docs.vllm.ai/en/latest/ If you are not sure, take llama.cpp. Swapping later is one command. 2. Download a coding model BEFORE you arrive. The venue wifi will not enjoy 50 people pulling several GB at 20:15, and this is the single most common reason somebody spends the lab hour watching a progress bar. A 7B-class coder model is the right size for this session. Qwen2.5-Coder-7B- Instruct is what the slides use, but bring whatever you actually want fast. Grab a ready-made Q4_K_M .gguf if you want to skip the conversion step, or the full weights if you want to do the shrinking yourself in step 2. 3. Confirm it runs at all. One reply, any speed, is enough: ./llama-server -m .gguf --host 0.0.0.0 --port 8080 then in another window: curl -s localhost:8080/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model":"local","messages":[{"role":"user","content":"hello"}]}' Slow is FINE. Slow is the entire premise of the evening. Broken is not. 4. Have one real coding task in mind. Something small from your own work, not a puzzle. You will run it hosted at 19:55 and against your own endpoint at 21:15, and the difference between those two numbers is what you take home. WHAT WE PROVIDE --------------- - z.ai with Claude Code for the session. Also what we clock at 19:55 to set the speed bar everything is measured against. - Nebius Token Factory credits for the lab hour, so you can stand up a fast endpoint on a real GPU even if your laptop cannot. Both are included in every ticket. RUNNING ORDER ------------- 19:00 doors 19:30 walkthrough - where the time goes, shrinking, serving, the four knobs 19:55 we time the hosted model together and write the number on the board 20:15 hands-on hour, five steps 21:30 demos, numbers round the room, social 22:00 out TICKETS ------- $10 early, $20 at the door. Frontier Tower members free with code FTMEMBER. Reach the team directly if cost is the blocker. A NOTE ON THE COMMANDS IN THE SLIDES ------------------------------------ llama.cpp and vLLM both rename and deprecate server flags between releases. If a flag on a slide is rejected at startup, `--help` on the binary you actually installed is the truth. This is normal and it is not you doing something wrong. Two things we hit ourselves while rehearsing this on 2026-08-05, so you do not have to: WINDOWS + curl + JSON. PowerShell strips the quotes out of an inline JSON body, and llama-server answers with `parse error ... last read '{m'`. It looks like the server is broken. It is not. Put the body in a file and pass it with -d "@req.json": '{"model":"local","messages":[{"role":"user","content":"hi"}],"max_tokens":64}' | Out-File req.json -Encoding ascii curl.exe -s -X POST http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d "@req.json" DO NOT RUN THE SERVER AS A BACKGROUND SERVICE TONIGHT. On Windows, a llama-server launched from a scheduled task silently fails to put the model on the graphics card, even with -ngl 99, and there is no error to tell you. Measured on the same machine, same model, same flags: 159 tokens/sec from a terminal versus 17.6 as a scheduled task. Keep it in a terminal window. Vibe Coding Nights. Facilitator Rayyan Zahid (Immersive Commons), with Michalis Vasileiadis (Hacker Bob), Eric Mockler (AI Geneticist) and Devinder Sodhi (Learning Layer Labs). Telegram: https://t.me/+EBFzKXmJAVk5ZGU0 Site: https://vibecodingnights.com