Skip to main content

Custom instruction issues

Connect docs via MCP

This guide covers four common behavioral symptoms when custom instructions for conversational AI don't work as expected: instructions being ignored, answers degrading after a change, replies that skip catalog search, and behavior that differs across surfaces.

Instruction seems to be ignored​

Symptom

Conversational AI continues answering as if the instruction doesn't exist, even after saving.

Cause​

The instruction references metadata that doesn't exist on your assets, or the change wasn't picked up by an in-progress conversation.

Solution​

  1. Confirm the metadata the instruction depends on is actually populated on your assets. Instructions that filter on attributes your catalog doesn't carry are silently skipped.
  2. Open a new conversation. Changes to custom instructions apply only to conversations started after saving—an in-progress conversation keeps the instructions it started with.

Answers degraded after recent changes​

Symptom

Answers became less relevant or less accurate after saving a new instruction.

Cause​

A recently added instruction is conflicting with other rules, duplicating access control logic, or encoding preferences that only apply to one team.

Solution​

  1. Remove the most recently added instruction and re-test with a new conversation.
  2. Check for routing or handoff logic first—this is the most common cause of broad degradation. See What to avoid.
  3. Introduce rules one at a time so regressions are easy to trace.

Conversational AI replies without searching​

Symptom

Conversational AI announces a skill or tool and then stops, without searching your catalog.

Cause​

Routing or orchestration logic in your instructions is telling conversational AI to announce a skill and stop, rather than answering the question.

Solution​

Remove any instructions that tell conversational AI to classify a question and hand it off to a named skill. Conversational AI selects skills itself as part of answering—instructions that try to drive that step produce replies that announce a skill and then stop. See What to avoid.

Behavior differs between Atlan and Slack or Microsoft Teams​

Symptom

The same question returns different answers depending on the surface—Atlan, Slack, or Microsoft Teams.

Solution​

Confirm which surfaces your tenant has enabled, and compare the same question in each. See Use conversational AI in your tools.

See also​

Need help​

If you need assistance after trying these steps, contact Atlan support.