Contributing to AgentOps

Thanks for checking out AgentOps. We're building tools to help developers like you make AI agents that actually work reliably. If you've ever tried to build an agent system, you know the pain - they're a nightmare to debug, impossible to monitor, and when something goes wrong... good luck figuring out why.

We created AgentOps to solve these headaches, and we'd love your help making it even better. Our SDK hooks into all the major Python frameworks (AG2, CrewAI, LangChain) and LLM providers (OpenAI, Anthropic, Cohere, etc.) to give you visibility into what your agents are actually doing.

How You Can Help

There are tons of ways to contribute, and we genuinely appreciate all of them:

Add More Providers: Help us support new LLM providers. Each one helps more developers monitor their agents.
Improve Framework Support: Using a framework we don't support yet? Help us add it!
Make Docs Better: Found our docs confusing? Help us fix them! Clear documentation makes everyone's life easier.
Share Your Experience: Using AgentOps? Let us know what's working and what isn't. Your feedback shapes our roadmap.

Even if you're not ready to contribute code, we'd love to hear your thoughts. Drop into our Discord, open an issue, or start a discussion. We're building this for developers like you, so your input matters.

Getting Started
Development Environment
Testing
Adding LLM Providers
Code Style
Pull Request Process
Documentation

Getting Started

Fork and Clone: First, fork the repository by clicking the 'Fork' button in the top right of the AgentOps repository. This creates your own copy of the repository where you can make changes.

Then clone your fork:
```
git clone https://github.com/YOUR_USERNAME/agentops.git
cd agentops
```
Add the upstream repository to stay in sync:
```
git remote add upstream https://github.com/AgentOps-AI/agentops.git
git fetch upstream
```
Before starting work on a new feature:
```
git checkout main
git pull upstream main
git checkout -b feature/your-feature-name
```
Install Dependencies:
```
pip install -e .
```
Set Up Pre-commit Hooks:
```
pre-commit install
```

Development Environment

Environment Variables: Create a .env file:

AGENTOPS_API_KEY=your_api_key
OPENAI_API_KEY=your_openai_key  # For testing
ANTHROPIC_API_KEY=your_anthropic_key  # For testing
# Other keys...

Virtual Environment: We recommend using poetry or venv:

python -m venv venv
source venv/bin/activate  # Unix
.\venv\Scripts\activate   # Windows

Pre-commit Setup: We use pre-commit hooks to automatically format and lint code. Set them up with:
```
pip install pre-commit
pre-commit install
```
That's it! The hooks will run automatically when you commit. To manually check all files:
```
pre-commit run --all-files
```

Testing

We use a comprehensive testing stack to ensure code quality and reliability. Our testing framework includes pytest and several specialized testing tools.

Testing Dependencies

Install all testing dependencies:

pip install -e ".[dev]"

We use the following testing packages:

pytest==7.4.0: Core testing framework
pytest-depends: Manage test dependencies
pytest-asyncio: Test async code
pytest-vcr: Record and replay HTTP interactions
pytest-mock: Mocking functionality
pyfakefs: Mock filesystem operations
requests_mock==1.11.0: Mock HTTP requests

Using Tox

We use tox to automate and standardize testing. Tox:

Creates isolated virtual environments for testing
Tests against multiple Python versions (3.7-3.12)
Runs all test suites consistently
Ensures dependencies are correctly specified
Verifies the package installs correctly

Run tox:

tox

This will:

Create fresh virtual environments
Install dependencies
Run pytest with our test suite
Generate coverage reports

Running Tests

Run All Tests:
```
tox
```
Run Specific Test File:
```
pytest tests/llms/test_anthropic.py -v
```
Run with Coverage:
```
coverage run -m pytest
coverage report
```

Writing Tests

Test Structure:

import pytest
from pytest_mock import MockerFixture
from unittest.mock import Mock, patch

@pytest.mark.asyncio  # For async tests
async def test_async_function():
    # Test implementation

@pytest.mark.depends(on=['test_prerequisite'])  # Declare test dependencies
def test_dependent_function():
    # Test implementation

Recording HTTP Interactions:

@pytest.mark.vcr()  # Records HTTP interactions
def test_api_call():
    response = client.make_request()
    assert response.status_code == 200

Mocking Filesystem:

def test_file_operations(fs):  # fs fixture provided by pyfakefs
    fs.create_file('/fake/file.txt', contents='test')
    assert os.path.exists('/fake/file.txt')

Mocking HTTP Requests:

def test_http_client(requests_mock):
    requests_mock.get('http://api.example.com', json={'key': 'value'})
    response = make_request()
    assert response.json()['key'] == 'value'

Testing Best Practices

Test Categories:
- Unit tests: Test individual components
- Integration tests: Test component interactions
- End-to-end tests: Test complete workflows
- Performance tests: Test response times and resource usage

Fixtures: Create reusable test fixtures in conftest.py:

@pytest.fixture
def mock_llm_client():
    client = Mock()
    client.chat.completions.create.return_value = Mock()
    return client

Test Data:
- Store test data in tests/data/
- Use meaningful test data names
- Document data format and purpose
VCR Cassettes:
- Store in tests/cassettes/
- Sanitize sensitive information
- Update cassettes when API changes

CI Testing Strategy

We use Jupyter notebooks as integration tests for LLM providers. This approach:

Tests real-world usage patterns
Verifies end-to-end functionality
Ensures examples stay up-to-date
Tests against actual LLM APIs

Notebook Tests:
- Located in examples/ directory
- Each LLM provider has example notebooks
- CI runs notebooks on PR merges to main
- Tests run against multiple Python versions
Test Workflow: The test-notebooks.yml workflow:
```
name: Test Notebooks
on:
  pull_request:
    paths:
      - "agentops/**"
      - "examples/**"
      - "tests/**"
```
- Runs on PR merges and manual triggers
- Sets up environment with provider API keys
- Installs AgentOps from main branch
- Executes each notebook
- Excludes specific notebooks that require manual testing
Provider Coverage: Each provider should have notebooks demonstrating:
- Basic completion calls
- Streaming responses
- Async operations (if supported)
- Error handling
- Tool usage (if applicable)
Adding Provider Tests:
- Create notebook in examples/provider_name/
- Include all provider functionality
- Add necessary secrets to GitHub Actions
- Update exclude_notebooks in workflow if manual testing needed

Adding LLM Providers

The agentops/llms/ directory contains provider implementations. Each provider must:

Inherit from BaseProvider:

@singleton
class NewProvider(BaseProvider):
    def __init__(self, client):
        super().__init__(client)
        self._provider_name = "ProviderName"

Implement Required Methods:
- handle_response(): Process LLM responses
- override(): Patch the provider's methods
- undo_override(): Restore original methods
Handle Events: Track:
- Prompts and completions
- Token usage
- Timestamps
- Errors
- Tool usage (if applicable)

Example Implementation Structure:

def handle_response(self, response, kwargs, init_timestamp, session=None):
    llm_event = LLMEvent(init_timestamp=init_timestamp, params=kwargs)
    try:
        # Process response
        llm_event.returns = response.model_dump()
        llm_event.prompt = kwargs["messages"]
        # ... additional processing
        self._safe_record(session, llm_event)
    except Exception as e:
        self._safe_record(session, ErrorEvent(trigger_event=llm_event, exception=e))

Code Style

Formatting:
- Use Black for Python code formatting
- Maximum line length: 88 characters
- Use type hints
Documentation:
- Docstrings for all public methods
- Clear inline comments
- Update relevant documentation
Error Handling:
- Use specific exception types
- Log errors with meaningful messages
- Include context in error messages

Pull Request Process

Branch Naming:
- feature/description
- fix/description
- docs/description
Commit Messages:
- Clear and descriptive
- Reference issues when applicable
PR Requirements:
- Pass all tests
- Maintain or improve code coverage
- Include relevant documentation
- Update CHANGELOG.md if applicable
Review Process:
- At least one approval required
- Address all review comments
- Maintain PR scope

Documentation

Types of Documentation:
- API reference
- Integration guides
- Examples
- Troubleshooting guides
Documentation Location:
- Code documentation in docstrings
- User guides in docs/
- Examples in examples/
Documentation Style:
- Clear and concise
- Include code examples
- Explain the why, not just the what

Getting Help & Community

We encourage active community participation and are here to help!

Preferred Communication Channels

GitHub Issues & Discussions:
- Open an issue for:
  - Bug reports
  - Feature requests
  - Documentation improvements
- Start a discussion for:
  - Questions about usage
  - Ideas for new features
  - Community showcase
  - General feedback
Discord Community:
- Join our Discord server for:
  - Real-time help
  - Community discussions
  - Feature announcements
  - Sharing your projects
Contact Form:
- For private inquiries, use our contact form
- Please note that public channels are preferred for technical discussions

License

By contributing to AgentOps, you agree that your contributions will be licensed under the MIT License.

Contributing to AgentOps

How You Can Help

There are tons of ways to contribute, and we genuinely appreciate all of them:

Add More Providers: Help us support new LLM providers. Each one helps more developers monitor their agents.
Improve Framework Support: Using a framework we don't support yet? Help us add it!
Make Docs Better: Found our docs confusing? Help us fix them! Clear documentation makes everyone's life easier.
Share Your Experience: Using AgentOps? Let us know what's working and what isn't. Your feedback shapes our roadmap.

Getting Started
Development Environment
Testing
Adding LLM Providers
Code Style
Pull Request Process
Documentation

Getting Started

Fork and Clone: First, fork the repository by clicking the 'Fork' button in the top right of the AgentOps repository. This creates your own copy of the repository where you can make changes.

Then clone your fork:
```
git clone https://github.com/YOUR_USERNAME/agentops.git
cd agentops
```
Add the upstream repository to stay in sync:
```
git remote add upstream https://github.com/AgentOps-AI/agentops.git
git fetch upstream
```
Before starting work on a new feature:
```
git checkout main
git pull upstream main
git checkout -b feature/your-feature-name
```
Install Dependencies:
```
pip install -e .
```
Set Up Pre-commit Hooks:
```
pre-commit install
```

Development Environment

Environment Variables: Create a .env file:

AGENTOPS_API_KEY=your_api_key
OPENAI_API_KEY=your_openai_key  # For testing
ANTHROPIC_API_KEY=your_anthropic_key  # For testing
# Other keys...

Virtual Environment: We recommend using poetry or venv:

python -m venv venv
source venv/bin/activate  # Unix
.\venv\Scripts\activate   # Windows

Pre-commit Setup: We use pre-commit hooks to automatically format and lint code. Set them up with:
```
pip install pre-commit
pre-commit install
```
That's it! The hooks will run automatically when you commit. To manually check all files:
```
pre-commit run --all-files
```

Testing

We use a comprehensive testing stack to ensure code quality and reliability. Our testing framework includes pytest and several specialized testing tools.

Testing Dependencies

Install all testing dependencies:

pip install -e ".[dev]"

We use the following testing packages:

pytest==7.4.0: Core testing framework
pytest-depends: Manage test dependencies
pytest-asyncio: Test async code
pytest-vcr: Record and replay HTTP interactions
pytest-mock: Mocking functionality
pyfakefs: Mock filesystem operations
requests_mock==1.11.0: Mock HTTP requests

Using Tox

We use tox to automate and standardize testing. Tox:

Creates isolated virtual environments for testing
Tests against multiple Python versions (3.7-3.12)
Runs all test suites consistently
Ensures dependencies are correctly specified
Verifies the package installs correctly

Run tox:

tox

This will:

Create fresh virtual environments
Install dependencies
Run pytest with our test suite
Generate coverage reports

Running Tests

Run All Tests:
```
tox
```
Run Specific Test File:
```
pytest tests/llms/test_anthropic.py -v
```
Run with Coverage:
```
coverage run -m pytest
coverage report
```

Writing Tests

Test Structure:

import pytest
from pytest_mock import MockerFixture
from unittest.mock import Mock, patch

@pytest.mark.asyncio  # For async tests
async def test_async_function():
    # Test implementation

@pytest.mark.depends(on=['test_prerequisite'])  # Declare test dependencies
def test_dependent_function():
    # Test implementation

Recording HTTP Interactions:

@pytest.mark.vcr()  # Records HTTP interactions
def test_api_call():
    response = client.make_request()
    assert response.status_code == 200

Mocking Filesystem:

def test_file_operations(fs):  # fs fixture provided by pyfakefs
    fs.create_file('/fake/file.txt', contents='test')
    assert os.path.exists('/fake/file.txt')

Mocking HTTP Requests:

def test_http_client(requests_mock):
    requests_mock.get('http://api.example.com', json={'key': 'value'})
    response = make_request()
    assert response.json()['key'] == 'value'

Testing Best Practices

Test Categories:
- Unit tests: Test individual components
- Integration tests: Test component interactions
- End-to-end tests: Test complete workflows
- Performance tests: Test response times and resource usage

Fixtures: Create reusable test fixtures in conftest.py:

@pytest.fixture
def mock_llm_client():
    client = Mock()
    client.chat.completions.create.return_value = Mock()
    return client

Test Data:
- Store test data in tests/data/
- Use meaningful test data names
- Document data format and purpose
VCR Cassettes:
- Store in tests/cassettes/
- Sanitize sensitive information
- Update cassettes when API changes

CI Testing Strategy

We use Jupyter notebooks as integration tests for LLM providers. This approach:

Tests real-world usage patterns
Verifies end-to-end functionality
Ensures examples stay up-to-date
Tests against actual LLM APIs

Notebook Tests:
- Located in examples/ directory
- Each LLM provider has example notebooks
- CI runs notebooks on PR merges to main
- Tests run against multiple Python versions
Test Workflow: The test-notebooks.yml workflow:
```
name: Test Notebooks
on:
  pull_request:
    paths:
      - "agentops/**"
      - "examples/**"
      - "tests/**"
```
- Runs on PR merges and manual triggers
- Sets up environment with provider API keys
- Installs AgentOps from main branch
- Executes each notebook
- Excludes specific notebooks that require manual testing
Provider Coverage: Each provider should have notebooks demonstrating:
- Basic completion calls
- Streaming responses
- Async operations (if supported)
- Error handling
- Tool usage (if applicable)
Adding Provider Tests:
- Create notebook in examples/provider_name/
- Include all provider functionality
- Add necessary secrets to GitHub Actions
- Update exclude_notebooks in workflow if manual testing needed

Adding LLM Providers

The agentops/llms/ directory contains provider implementations. Each provider must:

Inherit from BaseProvider:

@singleton
class NewProvider(BaseProvider):
    def __init__(self, client):
        super().__init__(client)
        self._provider_name = "ProviderName"

Implement Required Methods:
- handle_response(): Process LLM responses
- override(): Patch the provider's methods
- undo_override(): Restore original methods
Handle Events: Track:
- Prompts and completions
- Token usage
- Timestamps
- Errors
- Tool usage (if applicable)

Example Implementation Structure:

def handle_response(self, response, kwargs, init_timestamp, session=None):
    llm_event = LLMEvent(init_timestamp=init_timestamp, params=kwargs)
    try:
        # Process response
        llm_event.returns = response.model_dump()
        llm_event.prompt = kwargs["messages"]
        # ... additional processing
        self._safe_record(session, llm_event)
    except Exception as e:
        self._safe_record(session, ErrorEvent(trigger_event=llm_event, exception=e))

Code Style

Formatting:
- Use Black for Python code formatting
- Maximum line length: 88 characters
- Use type hints
Documentation:
- Docstrings for all public methods
- Clear inline comments
- Update relevant documentation
Error Handling:
- Use specific exception types
- Log errors with meaningful messages
- Include context in error messages

Pull Request Process

Branch Naming:
- feature/description
- fix/description
- docs/description
Commit Messages:
- Clear and descriptive
- Reference issues when applicable
PR Requirements:
- Pass all tests
- Maintain or improve code coverage
- Include relevant documentation
- Update CHANGELOG.md if applicable
Review Process:
- At least one approval required
- Address all review comments
- Maintain PR scope

Documentation

Types of Documentation:
- API reference
- Integration guides
- Examples
- Troubleshooting guides
Documentation Location:
- Code documentation in docstrings
- User guides in docs/
- Examples in examples/
Documentation Style:
- Clear and concise
- Include code examples
- Explain the why, not just the what

Getting Help & Community

We encourage active community participation and are here to help!

Preferred Communication Channels

GitHub Issues & Discussions:
- Open an issue for:
  - Bug reports
  - Feature requests
  - Documentation improvements
- Start a discussion for:
  - Questions about usage
  - Ideas for new features
  - Community showcase
  - General feedback
Discord Community:
- Join our Discord server for:
  - Real-time help
  - Community discussions
  - Feature announcements
  - Sharing your projects
Contact Form:
- For private inquiries, use our contact form
- Please note that public channels are preferred for technical discussions

License

By contributing to AgentOps, you agree that your contributions will be licensed under the MIT License.

Contributing to AgentOps

Contributing to AgentOps

How You Can Help

Table of Contents

Getting Started

Development Environment

Testing

Testing Dependencies

Using Tox

Running Tests

Writing Tests

Testing Best Practices

CI Testing Strategy

Adding LLM Providers

Code Style

Pull Request Process

Documentation

Getting Help & Community

Preferred Communication Channels

License

Related Skills

<h1 align="center">

Frontend Typescript Linting.mdc

2. Apply Deepthink Protocol (reason about dependencies

Contributing to AgentOps

How You Can Help

Table of Contents

Getting Started

Development Environment

Testing

Testing Dependencies

Using Tox

Running Tests

Writing Tests

Testing Best Practices

CI Testing Strategy

Adding LLM Providers

Code Style

Pull Request Process

Documentation

Getting Help & Community

Preferred Communication Channels

License