For further context, I modified /examples/pg-vector-store/query.ts to test the difference between using streaming ContextChatEngine and some other method. This is using the example data, which has this snippet: There are several competitors in the battery manufacturing market with Duracell holding an approximately 29% market share of the global alkaline battery market

So a context model should be able to answer the question of Duracell's alkaline battery market at 29%.

Output from streaming chat with context retriever:
As of my last update in 2021, Duracell is one of the leading brands in the global alkaline battery market. However, specific market share figures can fluctuate year by year due to various factors such as competition, market conditions, and consumer preferences.

For the most accurate and up-to-date information on Duracell's market share in the global alkaline battery market, I recommend consulting recent market research reports or financial statements from Duracell's parent company, Berkshire Hathaway. Industry analysis firms like Nielsen, Statista, or Market Research Future often publish detailed reports that include market share data for major players in the battery industry.

Output from queryEngine:
Duracell holds an approximately 29% market share of the global alkaline battery market.

So we can see the contentEngine is not using the context from the above queries. Script used is here (after running /examples/pg-vector-store/load-docs.ts).

import {
  ContextChatEngine,
  OpenAI,
  PGVectorStore,
  Settings,
  VectorStoreIndex,
} from "llamaindex";

Settings.llm = new OpenAI({ model: "gpt-4o" });

async function main() {
  try {
    const pgvs = new PGVectorStore();
    // Optional - set your collection name, default is no filter on this field.
    // pgvs.setCollection();

    const messages = [
      {
        role: "user",
        content:
          "What is duracell's market share of the global alkaline battery market?",
      },
    ];

    const query = messages[messages.length - 1].content;

    const index = await VectorStoreIndex.fromVectorStore(pgvs);

    // Below using chatEngine
    const retriever = index.asRetriever({
      topK: {
        TEXT: 3,
        IMAGE: 3,
      },
    });
    const chatEngine = new ContextChatEngine({
      retriever,
    });

    console.log("Output from streaming chat with context retriever:");
    const stream = await chatEngine.chat({
      message: query,
      chatHistory: messages,
      stream: true,
      verbose: true,
    });

    for await (const chunk of stream) {
      process.stdout.write(chunk.response);
    }
    console.log("\n");

    // Below using queryEngine
    const queryEngine = await index.asQueryEngine();
    const response = await queryEngine.query({ query });
    console.log("Output from queryEngine:");
    console.log(response.message.content);
  } catch (err) {
    console.error(err);
  }
}

main()
  .catch(console.error)
  .finally(() => {
    process.exit(1);
  });

what nodes do you used when call query? I'm guessing it's because there's no nodes

Code for both .query and .chat is above. Both use the same VectorStoreIndex from PGVectorStore which I generated by following the examples in examples/pg-vector-store/README.md. So I think they should be using the same nodes?

Seems like it's an issue with chatHistory.getMessages in the chat engine - it isn't returning the context, just the chat history messages.

Thanks for debugging. I will check later

For now I don't have much time to create a minumal reprodue code. But I think this might related to Memory refactor or it's already a bug from very early commits. We lack some test cases here, so you can provide more detail expected behavior to this ticket and I will add to my todo list

@marcusschiesser could you please help me on this?

@marcusschiesser sure, i'll have a look