Building a Continuous AI Workflow with PostHog and GitHub

Build an automated system that continuously monitors PostHog analytics, analyzes user behavior with AI, and creates GitHub issues automatically using PostHog MCP.

What You'll Build

A fully automated workflow that uses Continue CLI with the PostHog MCP to fetch analytics data, analyze user experience issues with AI, and automatically create GitHub issues with the GitHub CLI.

What You’ll Learn

This cookbook teaches you to:

• Use PostHog MCP to query analytics, errors, and feature flags
• Analyze user behavior patterns with AI
• Automatically create GitHub issues using GitHub CLI
• Set up continuous monitoring with GitHub Actions

Prerequisites

Before starting, ensure you have:

• GitHub repository where you want to create issues
• PostHog account with session recordings enabled and data collecting
• Node.js 18+ installed locally
• Continue CLI with active credits (required for API usage)
• GitHub CLI installed (gh command)

1. Install Continue CLI

   npm i -g @continuedev/cli
   

2. Set up Continue CLI Account & API Key

   1. Visit Continue Organizations
   2. Sign up or log in to your Continue account
   3. Navigate to your organization settings
   4. Click “API Keys” and then ”+ New API Key”
   5. Copy the API key immediately (you won’t see it again!)
   6. Login to the CLI: cn login
3. Add Required Secrets to Continue CLI

   Continue CLI will securely store your API keys as secrets that can be referenced in prompts.

Continue CLI handles the complex API interactions - you just need to provide the right prompts!

Step 1: Set Up Your Credentials

First, you’ll need to gather your PostHog and GitHub API credentials and add them as secrets in Continue CLI.

PostHog API Credentials

You’ll need a Personal API Key (not a Project API key) to access session recordings:

1. Go to Personal API Keys in PostHog
2. Click + Create a personal API Key
3. Name it “Continue CLI Session Analysis”
4. Select these scopes:
   • session_recording:read - Required for accessing session data
   • insight:read
   • query:read
   • session_recording_playlist:read
5. Copy the key immediately (you won’t see it again!)
6. Note your Project ID from your PostHog project settings
7. Note your PostHog host URL (e.g., https://us.posthog.com or your custom domain)
8. You’ll also need your POSTHOG_AUTH_HEADER value, which is simply Bearer YOUR_API_KEY

Continue Secrets: The POSTHOG_AUTH_HEADER secret should be stored in Continue’s secure secrets storage. This keeps your API key safe and the MCP automatically connects to your default PostHog project.

PostHog GitHub Continuous AI Workflow Options

🚀 Fastest Path to Success

Skip the manual setup and use our pre-built PostHog GitHub agent that includes optimized prompts, rules, and the PostHog MCP for more consistent results.

How PostHog MCP Works:

• Your API key is tied to your PostHog account and organization
• It automatically uses your default project (no project ID needed)
• If you have multiple projects, use mcp__posthog__switch-project to change
• The MCP connects via https://mcp.posthog.com/sse using your account context.

⚡ Quick Start (Recommended)

Perfect for: Immediate results with optimized prompts and built-in debugging

1. Add the Pre-Built Agent

   Visit the PostHog GitHub Continuous AI Agent on Continue Hub and click “Install Agent” or run:

   cn --config continuedev/awesome-models-posthog-gh
   

   This agent includes:

   • Optimized prompts for PostHog analysis and GitHub issue creation
   • Built-in rules for consistent formatting and error handling
   • PostHog MCP for more reliable API interactions
2. Run the Analysis

   Navigate to your project directory and run:

   cn "Give me my PostHog Session data and create GitHub issues based on the problems."
   

   That’s it! The agent handles everything automatically.

Why Use the Agent? Results are more consistent and debugging is easier thanks to the PostHog MCP integration and pre-tested prompts.

Why GitHub CLI over GitHub MCP: While GitHub MCP is available, it can be token-expensive to run. The gh CLI is more efficient, requires no API tokens (authenticated via gh auth login), and provides a cleaner command-line experience. GitHub MCP remains an option if you prefer full MCP integration.

Agent Requirements

To use the pre-built agent, you need either:

• Continue CLI Pro Plan with the models add-on, OR
• Your own API keys added to Continue Hub secrets (same as Step 1 below) The agent will automatically detect and use your configuration.

---

Repository Labels Required: Make sure your GitHub repository has these labels:

• bug, enhancement
• high-priority, medium-priority, low-priority
• user-experience, automated

Create missing labels in your repo at: Settings → Labels → New label

What Continue CLI Does: - Parses your analysis results automatically - Makes authenticated GitHub API calls using your stored token - Creates properly formatted issues with appropriate labels - Checks for duplicate issues to avoid spam - Provides confirmation with issue URLs

Setting Up Automated GitHub Actions Workflow

Create a GitHub Actions workflow that runs your Continue CLI analysis automatically on a schedule.

Add GitHub Secrets

First, add these secrets to your GitHub repository:

1. Go to Repository Settings → Secrets and variables → Actions
2. Click “New repository secret” and add:
   • CONTINUE_API_KEY: Your Continue API key from hub.continue.dev

Create GitHub Actions Workflow

Create .github/workflows/posthog-analysis.yml in your repository:

⚡ Using Pre-Built Agent (Recommended)

name: PostHog Session Analysis with Continue CLI Agent

on:
schedule: - cron: "0 6 \* \* \*" # Run at 6 AM UTC daily
workflow_dispatch: # Allow manual triggering

jobs:
analyze:
runs-on: ubuntu-latest

    permissions:
      contents: read
      issues: write

    steps:
      - uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "18"

      - name: Install Continue CLI
        run: |
          npm install -g @continuedev/cli
          echo "✅ Continue CLI installed"
          cn --version

      - name: Authenticate with Continue CLI
        run: |
          # Set Continue API key
          export CONTINUE_API_KEY="${{ secrets.CONTINUE_API_KEY }}"
          # Login to Continue CLI
          if ! cn auth login --api-key "$CONTINUE_API_KEY"; then
            echo "❌ Failed to authenticate with Continue CLI"
            exit 1
          fi
          echo "✅ Continue CLI authenticated"

      - name: Load PostHog GitHub Agent
        run: |
          echo "🤖 Loading PostHog GitHub Continuous AI Agent..."
          cn --config continuedev/awesome-models-posthog-gh
          echo "✅ Agent loaded successfully"

      - name: Run PostHog Analysis and Create GitHub Issues
        run: |
          echo "🎬 Starting automated PostHog session analysis..."

          # Run the agent with the simple prompt
          cn -p "Tell me about my PostHog session recordings and create GitHub issues based on the problems."

          echo "✅ Analysis and issue creation completed"

      - name: Summary
        run: |
          echo "✅ PostHog analysis workflow completed!"
          echo "📊 Check the artifacts for detailed analysis results"
          echo "🐛 Check your repository issues for newly created UX issues"
          echo "🤖 Agent automatically handled both analysis and issue creation"

Test the GitHub Actions Workflow

1. Manual Trigger: Go to Actions tab in your repository and manually trigger the workflow
2. Check Artifacts: Download the analysis results from the workflow run
3. Verify Issues: Check that GitHub issues were created in your repository

Success Indicators:

• PostHog API returns session data (not empty)
• Continue CLI generates analysis with identified issues
• GitHub issues are created with proper labels and formatting
• Workflow completes without errors

What You’ve Built

After completing this guide, you have a complete Continuous AI system that:

• Monitors user experience - Automatically fetches and analyzes PostHog session data

• Identifies problems intelligently - Uses AI to spot patterns and technical issues

• Creates actionable tasks - Generates GitHub issues with specific recommendations

• Runs autonomously - Operates daily without manual intervention using GitHub Actions

• Scales with your team - Handles growing amounts of session data automatically

Continuous AI

Your system now operates at Level 2 Continuous AI - AI handles routine analysis tasks with human oversight through GitHub issue review and prioritization.

Security Best Practices

Protect Your API Keys:

• Store all credentials as GitHub Secrets, never in code
• Use Continue CLI’s secure secret storage
• Limit token scopes to minimum required permissions
• Rotate API keys regularly (every 90 days recommended)
• Monitor token usage for unusual activity

Advanced Prompts

Consider enhancing your workflow with these advanced Continue CLI prompts:

Performance Analysis

“Analyze PostHog performance metrics alongside session recordings to identify slow page loads affecting user experience”

Error Correlation

“Cross-reference JavaScript console errors with user actions to identify the root cause of UX issues”

Feature Flag Analysis

“Analyze how PostHog feature flags impact user behavior in session recordings”

Custom Slack Alerts

“Create Slack alerts when critical UX issues are detected in PostHog sessions”

Next Steps

• Consider GitHub MCP as an alternative (note: can be token-expensive)
• Configure Slack MCP for alerts
• Set up PostHog performance monitoring
• Join the Continue Discord for support

Resources

• PostHog API Documentation
• PostHog MCP Documentation
• PostHog Session Replay
• PostHog Feature Flags
• PostHog Error Tracking
• GitHub CLI Documentation
• GitHub MCP on Continue Hub (alternative option)
• Continue CLI Guide
• Continuous AI Best Practices