I’ve got 24GB of VRAM, which helps a lot. I’m using the Q4_K_M version of this model, with the KV cache quantized to Q8, and that’s able to get me to 128000 tokens of context. I’m using the Qwen Code harness in VS Code.
If 128000 tokens still isn’t enough, break the task down into smaller chunks and help the harness out with MCP tools - I installed Codebase Memory to help it navigate large codebases more efficiently, for example. Ask the coding agent what’s causing problems and it may be able to help identify solutions too, I’ve had it troubleshoot itself several times now and it’s been quite good at inspecting its own settings and finding where the problems are with them. I even installed Firecrawl for it so it can do its own web searches.
When you add the additional codebase memory, that will also add all those mcp tools to each round-trip to the llm model. Do you still see improvements and less filling up if your context? Just curious how it works out; I can imagine that a text based memory that explains the architecture of the codebase would help as well. Not sure about the difference if using a graph to store the code context?
Depends on the framework you’re using. I’m using Qwen Code Companion (via a VS Code IDE plugin) and it’s got some kind of tiered system built into it for adding tool definitions to context, it’s got a standard built-in set that’s always mentioned but other stuff only gets added to the context under certain circumstances. I can’t recall all the details, I found out about it when I was chatting with Qwen Code to ask it why it hadn’t used a particular MCP I’d given it to solve a problem despite it seeming like a useful tool for the task and we got into a discussion about to configure it. As I recall it’s something along the lines of most tools only being present in context by default as little one-liner stubs and the agent has to tell the framework “that one looks like it might be useful for what I’m doing” for the full definition to get loaded.
Most of the time I’m working on small scripts and specialized applications these days, and codebase memory is overkill for those so I haven’t actually seen it in use much.
I’ve got 24GB of VRAM, which helps a lot. I’m using the Q4_K_M version of this model, with the KV cache quantized to Q8, and that’s able to get me to 128000 tokens of context. I’m using the Qwen Code harness in VS Code.
If 128000 tokens still isn’t enough, break the task down into smaller chunks and help the harness out with MCP tools - I installed Codebase Memory to help it navigate large codebases more efficiently, for example. Ask the coding agent what’s causing problems and it may be able to help identify solutions too, I’ve had it troubleshoot itself several times now and it’s been quite good at inspecting its own settings and finding where the problems are with them. I even installed Firecrawl for it so it can do its own web searches.
@FaceDeer @eicker @floofloof @fonix232 @naeap
When you add the additional codebase memory, that will also add all those mcp tools to each round-trip to the llm model. Do you still see improvements and less filling up if your context? Just curious how it works out; I can imagine that a text based memory that explains the architecture of the codebase would help as well. Not sure about the difference if using a graph to store the code context?
Depends on the framework you’re using. I’m using Qwen Code Companion (via a VS Code IDE plugin) and it’s got some kind of tiered system built into it for adding tool definitions to context, it’s got a standard built-in set that’s always mentioned but other stuff only gets added to the context under certain circumstances. I can’t recall all the details, I found out about it when I was chatting with Qwen Code to ask it why it hadn’t used a particular MCP I’d given it to solve a problem despite it seeming like a useful tool for the task and we got into a discussion about to configure it. As I recall it’s something along the lines of most tools only being present in context by default as little one-liner stubs and the agent has to tell the framework “that one looks like it might be useful for what I’m doing” for the full definition to get loaded.
Most of the time I’m working on small scripts and specialized applications these days, and codebase memory is overkill for those so I haven’t actually seen it in use much.