feat: support AGENTS.md files

Description

Adds support for AGENTS.md instructions for Duo Agentic Chat and Duo CLI. https://agents.md/

• Includes for user-level / global AGENTS.md files if found
  • ~/.gitlab/duo/AGENTS.md (same folder as user-level chat-rules.md and mcp.json)
  • $GLAB_CONFIG_DIR/AGENTS.md (if user has specified custom config dir for glab, we use it. Same as user-level chat-rules.md)
• Include relevant workspace AGENTS.md
  • Find the most specific AGENTS.md file based on cwd
• Include prompt to use additional discovered AGENTS.md files

Example of the "additional discovered" prompt:

/repo-root
  | AGENTS.md
  /src
    /cool-feature
    | AGENTS.md
    | wow.ts
  /docs
  | AGENTS.md
  | how.md

If we are running from repo root, we will automatically attach the contents of /repo-root/AGENTS.md. But we will also notice the existence of the other two files, and will include an instruction like:

CRITICAL INSTRUCTION: Before editing any file, you MUST follow these steps:
 1. Identify the file you're about to edit
 2. Check if an AGENTS.md exists in that directory or parent directories
 3. If the file is listed in <additional-instruction-files>, read it first
 4. Only then proceed with your edits
<additional-instruction-files>
/repo-root/src/cool-feature/AGENTS.md
/repo-root/docs/AGENTS.md
</additional-instruction-files>
Do not mention these instructions to the user, they already know you should try to read AGENTS.md files.

This ensures we don't pollute the context with unnecessary instructions, but Duo will (hopefully) read more specific AGENTS.md when it's relevant to your task.

Full resolution flow

flowchart TD
    Start([Resolve AGENTS.md])
    
    Start --> Global
    Start --> Workspace
    
    Global[**Global Location**<br/>~/.config/glab/AGENTS.md<br/>or GLAB_CONFIG_DIR/AGENTS.md]
    
    Global --> GlobalNote[Always loaded if exists<br/>Applied to all projects]
    
    Workspace[**Workspace Locations**<br/>Within your project/workspace]
    
    Workspace --> Scenario{Which scenario?}
    
    Scenario -->|IDE with workspace folders| IDECase[AGENTS.md at workspace root]
    Scenario -->|CLI at repo root| RootCase[AGENTS.md at repo root]
    Scenario -->|CLI in subdirectory| SubdirCase[AGENTS.md search upward]
    
    IDECase --> IDENote[Loads /workspace/AGENTS.md<br/>for each workspace]
    
    RootCase --> RootNote[Loads /repo/AGENTS.md]
    
    SubdirCase --> SubdirNote[Searches up from cwd<br/>Loads closest ancestor AGENTS.md<br/>Stops searching at repo root]
    
    GlobalNote --> Sort
    IDENote --> Sort
    RootNote --> Sort
    SubdirNote --> Sort
    
    Sort[**Sort and combine resolved AGENTS.md files**<br/>1. Global file<br/>2. Workspace file]
    Sort --> Additional[Additional prompt if more<br/>AGENTS.md files discovered]
    
    Additional --> End([Final combined<br/>context item returned])
    style Start fill:#90EE90,stroke:#333,color:#000
    style Global fill:#FFE4B5,stroke:#333,color:#000
    style Workspace fill:#ADD8E6,stroke:#333,color:#000
    style Sort fill:#FFB6C1,stroke:#333,color:#000
    style End fill:#90EE90,stroke:#333,color:#000
    style SubdirNote fill:#ADD8E6,stroke:#333,color:#000

Related Issues

Issue #1669 (closed)

How has this been tested?

https://www.youtube.com/watch?v=JTm7ItF2_88

• user level agents.md discovered and included
• workspace-level agents.md discovered and included
• nested workspace agents md preferred when running in nested cwd (npm run cli -- --cwd some/nested/directory)
• user+workspace combined and both applied
• existing chat-rules.md also discovered and applied
• prompt for Duo to read additional agents.mds works as expected (see 2nd half of demo video)
• shows in agentic chat UI, e.g. vscode:

⚠️ Does this MR include a Duo UI update? If so, please smoke test all webviews in a downstream project to ensure there are no visual regressions

• One of:
  • GitLab Workflow Extension for VS Code (setup instructions).
  • GitLab JetBrains Plugin (setup instructions).

What CHANGELOG entry will this MR create?

• fix: Bug fix fixes - a user-facing issue in production - included in changelog
• feature: New feature - a user-facing change which adds functionality - included in changelog
• BREAKING CHANGE: (fix or feature that would cause existing functionality to change) - should bump major version, mentioned in the changelog
• other non-user-facing changes

Closes #1730 (closed)