While building a research agent, i realized the models wasn’t generating exactly what i want, so i thought of solving that in a way where the prompt sent to the model should contain all that is needed to generate the preferred result, but this can't be stuffed in the prompt every turn/workflow, so, intent and perception that makes itself better should be added to the prompt too before sending it too the LLM and of course these intent and perception are formulated heuristically is used to understand and the users' style, intent and preference over time.

HOW DID I SOLVE IT?

Using Pydantic as the core. Ingest → Chunk → Embed → Index → Retrieve → Rerank → Assemble → Deliver the prompt is ingested and then goes through the vector system local ONNX MiniLM embeddings + JSON/SQLite chunk storage + brute-force cosine search, blended with lexical/keyword scoring during the retrieval.

The intents are programmed in these two manners: Query-time intent; intent_data + intent_schema and Derived workflow intent; ContextBucketWorkflowIntent
Preference is programmed: Preferences live as normal context-bucket records with a declared schema:

• schema_name: "user_workflow_preference"

• source_key: defaults to "user_workflow_preference" (configurable)

• scope: usually "user" (can be session-scoped)

• structured_data: the preference fields as JSON
  You can set them explicitly via update-workflow-preference / service.update_workflow_preference(), which merges new data onto any existing profile.

Both the intent and the preference come together in the task envelope prepare_task_envelope() is the main integration point. Also, task enveloped output to LLM results are also being sent to the bucket to be used to prune preferences for immediate or long-term workflows.

The beauty of this is that it is 100% local to keep data safe and not feed models sensitive information. The constraint here is that the embedding system isn’t perfect, hence the benchmark results are not impressive, but this has been tested with a standard vector system, and the result was way above expectations.

Repository Link: https://github.com/crackdevbuild/context-bucket