gurix/clubber
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add OpenSpec proposal for pytest test suite

Browse Source 
This proposal adds comprehensive pytest testing for all existing features:
- Unit tests for validation logic
- Integration tests for database operations and GraphQL resolvers
- E2E tests for complete API flows
- MCP server tests for all 6 tools

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
gurix
2025-11-21 13:56:38 +01:00
co-authored by Claude
parent dd772db298
commit e041684acd
3 changed files with 488 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
openspec/changes/add-pytest-tests/proposal.md
+91
View File
@@ -0,0 +1,91 @@
# Add Pytest Tests Proposal
## Why
The project currently has no automated tests despite having pytest and pytest-asyncio configured in `pyproject.toml`. This creates several risks:
- **Regression risk**: Changes to existing features (member CRUD operations, GraphQL resolvers, MCP tools, validation logic) may break functionality without detection
- **Confidence**: Developers and AI assistants lack confidence when refactoring or adding features
- **Documentation gap**: Tests serve as executable documentation of expected behavior
- **Quality assurance**: No automated verification of business rules (validation, sorting, error handling)
Testing is critical for:
1. Member CRUD operations (create, read, update, delete)
2. GraphQL API (queries, mutations, introspection)
3. MCP server tools (6 tools connecting to GraphQL API)
4. Validation logic (email, phone, firstName)
5. Database operations (async SQLAlchemy patterns)
## What
Implement comprehensive pytest test suite covering existing features:
### Test Infrastructure
- Create `tests/` directory with proper structure (unit, integration, e2e, mcp)
- Configure pytest fixtures for database setup (in-memory SQLite)
- Configure async test support (pytest-asyncio already in dependencies)
- Add conftest.py with shared fixtures
### Test Coverage
1. **Unit Tests** (`tests/unit/`)
- Validation logic (email, phone, firstName)
- Utility functions
2. **Integration Tests** (`tests/integration/`)
- Database operations with Member model
- GraphQL resolvers (Query, Mutation)
- Error handling (MemberNotFoundError, ValidationError)
3. **E2E Tests** (`tests/e2e/`)
- Complete GraphQL query flows
- Complete GraphQL mutation flows
- GraphQL introspection queries
4. **MCP Tests** (`tests/mcp/`)
- MCP server tool definitions
- MCP tool execution (list_members, get_member, create_member, update_member)
- General GraphQL tools (get_graphql_schema, execute_graphql_query)
### Documentation
- Update README.md with testing instructions
- Add "Running Tests" section
- Add "Test Coverage" section
- Document test organization and conventions
## Impact
### Breaking Changes
None - purely additive.
### New Dependencies
None - pytest and pytest-asyncio already in dev dependencies.
### Migration
No migration needed.
## Alternatives Considered
1. **unittest instead of pytest**
- Rejected: pytest is already configured and provides better async support, fixtures, and test discovery
2. **Test only critical paths**
- Rejected: Comprehensive testing provides better confidence and documentation
3. **Skip MCP tests**
- Rejected: MCP server is a core component and needs testing
## Success Criteria
1. Test suite runs with `uv run pytest`
2. All existing features have test coverage
3. Tests are organized logically (unit, integration, e2e, mcp)
4. Tests use proper fixtures for database setup
5. README.md documents how to run tests
6. No changes to production code (except minor fixes if bugs are found)
## Related Changes
None - this is the first testing implementation.
## Open Questions
None - the implementation is straightforward.
openspec/changes/add-pytest-tests/specs/project-setup/spec.md
+295
View File
@@ -0,0 +1,295 @@
# project-setup Spec Delta
## ADDED Requirements
### Requirement: Automated Test Suite with Pytest
The project SHALL provide a comprehensive test suite using pytest to verify all features and prevent regressions.
#### Scenario: Run test suite successfully
- **GIVEN** the project is set up with dependencies installed
- **WHEN** the developer runs `uv run pytest`
- **THEN** all tests SHALL execute
- **AND** test results SHALL be displayed with pass/fail status
- **AND** the command SHALL exit with code 0 if all tests pass
#### Scenario: Run tests with verbose output
- **GIVEN** the project has tests implemented
- **WHEN** the developer runs `uv run pytest --verbose`
- **THEN** detailed test output SHALL be displayed
- **AND** each test name and result SHALL be shown
#### Scenario: Run specific test categories
- **GIVEN** tests are organized in subdirectories
- **WHEN** the developer runs `uv run pytest tests/unit`
- **THEN** only unit tests SHALL execute
- **WHEN** the developer runs `uv run pytest tests/integration`
- **THEN** only integration tests SHALL execute
### Requirement: Test Organization Structure
Tests SHALL be organized in a clear directory structure separating different test types.
#### Scenario: Test directory structure exists
- **GIVEN** the project repository
- **WHEN** inspecting the directory structure
- **THEN** a `tests/` directory SHALL exist at the project root
- **AND** subdirectories `tests/unit/`, `tests/integration/`, `tests/e2e/`, and `tests/mcp/` SHALL exist
- **AND** a `tests/conftest.py` file SHALL exist for shared fixtures
#### Scenario: Test files follow naming convention
- **GIVEN** test files in the `tests/` directory
- **WHEN** examining file names
- **THEN** all test files SHALL be named `test_*.py`
- **AND** test file names SHALL correspond to the modules they test (e.g., `test_validation.py` for `validation.py`)
### Requirement: Shared Test Fixtures
The test suite SHALL provide reusable fixtures for common test setup via conftest.py.
#### Scenario: Database fixture provides in-memory SQLite
- **GIVEN** tests need database access
- **WHEN** a test function requests the `db` fixture
- **THEN** an in-memory SQLite database SHALL be created
- **AND** the database SHALL have all tables created via Alembic migrations or direct schema creation
- **AND** the database SHALL be cleaned up after the test completes
#### Scenario: Async session fixture provides database sessions
- **GIVEN** tests need to interact with the database
- **WHEN** a test function requests the `async_session` fixture
- **THEN** an async SQLAlchemy session SHALL be provided
- **AND** the session SHALL be connected to the test database
- **AND** the session SHALL be closed after the test completes
#### Scenario: GraphQL client fixture provides test client
- **GIVEN** tests need to make GraphQL requests
- **WHEN** a test function requests the `graphql_client` fixture
- **THEN** a FastAPI test client SHALL be provided
- **AND** the client SHALL be configured to use the test database
- **AND** the client SHALL support async GraphQL queries and mutations
### Requirement: Unit Tests for Validation Logic
Unit tests SHALL verify validation functions work correctly with valid and invalid inputs.
#### Scenario: Test firstName validation with valid input
- **GIVEN** a valid firstName "Alice"
- **WHEN** `validate_first_name("Alice")` is called
- **THEN** no exception SHALL be raised
#### Scenario: Test firstName validation with empty string
- **GIVEN** an empty firstName ""
- **WHEN** `validate_first_name("")` is called
- **THEN** a ValidationError SHALL be raised
- **AND** the error message SHALL indicate firstName cannot be empty
#### Scenario: Test email validation with valid email
- **GIVEN** a valid email "test@example.com"
- **WHEN** `validate_email("test@example.com")` is called
- **THEN** no exception SHALL be raised
#### Scenario: Test email validation with invalid email
- **GIVEN** an invalid email "not-an-email"
- **WHEN** `validate_email("not-an-email")` is called
- **THEN** a ValidationError SHALL be raised
- **AND** the error message SHALL indicate invalid email format
#### Scenario: Test email validation with None
- **GIVEN** email is None
- **WHEN** `validate_email(None)` is called
- **THEN** no exception SHALL be raised (None is allowed)
#### Scenario: Test phone validation with valid E.164 number
- **GIVEN** a valid phone "+41791234567"
- **WHEN** `validate_phone("+41791234567")` is called
- **THEN** no exception SHALL be raised
#### Scenario: Test phone validation with invalid number
- **GIVEN** an invalid phone "123"
- **WHEN** `validate_phone("123")` is called
- **THEN** a ValidationError SHALL be raised
#### Scenario: Test phone validation with None
- **GIVEN** phone is None
- **WHEN** `validate_phone(None)` is called
- **THEN** no exception SHALL be raised (None is allowed)
### Requirement: Integration Tests for Member Model
Integration tests SHALL verify database operations with the Member model.
#### Scenario: Create member with minimal data
- **GIVEN** a test database session
- **WHEN** a Member is created with only firstName="Alice"
- **THEN** the member SHALL be persisted to the database
- **AND** the member SHALL have an auto-generated ID
- **AND** created_at and updated_at timestamps SHALL be populated
- **AND** all optional fields SHALL be None
#### Scenario: Create member with complete data
- **GIVEN** a test database session
- **WHEN** a Member is created with all fields populated
- **THEN** the member SHALL be persisted with all data
- **AND** all fields SHALL match the provided values
#### Scenario: Member timestamps are set correctly
- **GIVEN** a newly created member
- **THEN** created_at SHALL be set to current time
- **AND** updated_at SHALL be set to current time
- **AND** created_at SHALL equal updated_at for new members
### Requirement: Integration Tests for GraphQL Queries
Integration tests SHALL verify GraphQL query resolvers return correct data.
#### Scenario: Query single member by ID
- **GIVEN** a member exists with ID 1
- **WHEN** the `member(id: 1)` query is executed
- **THEN** the member data SHALL be returned
- **AND** all requested fields SHALL match the database record
#### Scenario: Query member with non-existent ID
- **GIVEN** no member exists with ID 999
- **WHEN** the `member(id: 999)` query is executed
- **THEN** None SHALL be returned
- **AND** no error SHALL be raised
#### Scenario: List all members
- **GIVEN** multiple members exist in the database
- **WHEN** the `members` query is executed
- **THEN** an array of all members SHALL be returned
- **AND** members SHALL be sorted by last_name (nulls last), then first_name
#### Scenario: List members when database is empty
- **GIVEN** no members exist in the database
- **WHEN** the `members` query is executed
- **THEN** an empty array SHALL be returned
### Requirement: Integration Tests for GraphQL Mutations
Integration tests SHALL verify GraphQL mutation resolvers create, update, and delete members correctly.
#### Scenario: Create member with minimal data
- **GIVEN** CreateMemberInput with only firstName="Bob"
- **WHEN** the `createMember` mutation is executed
- **THEN** a new member SHALL be created in the database
- **AND** the member SHALL have firstName="Bob"
- **AND** all optional fields SHALL be None
- **AND** the mutation SHALL return the created member with ID
#### Scenario: Create member with complete data
- **GIVEN** CreateMemberInput with all fields populated
- **WHEN** the `createMember` mutation is executed
- **THEN** a new member SHALL be created with all data
- **AND** all fields SHALL match the input
#### Scenario: Create member with invalid email
- **GIVEN** CreateMemberInput with email="not-an-email"
- **WHEN** the `createMember` mutation is executed
- **THEN** a ValidationError SHALL be raised
- **AND** no database record SHALL be created
#### Scenario: Update member successfully
- **GIVEN** a member exists with ID 1
- **WHEN** the `updateMember` mutation is executed with id=1 and email="new@example.com"
- **THEN** the member's email SHALL be updated to "new@example.com"
- **AND** the updated_at timestamp SHALL be refreshed
- **AND** all other fields SHALL remain unchanged
#### Scenario: Update member with non-existent ID
- **GIVEN** no member exists with ID 999
- **WHEN** the `updateMember` mutation is executed with id=999
- **THEN** a MemberNotFoundError SHALL be raised
- **AND** the error message SHALL indicate member not found
#### Scenario: Delete member successfully
- **GIVEN** a member exists with ID 1
- **WHEN** the `deleteMember` mutation is executed with id=1
- **THEN** the member SHALL be removed from the database
- **AND** the mutation SHALL return true
- **AND** subsequent queries for that member SHALL return None
#### Scenario: Delete member with non-existent ID
- **GIVEN** no member exists with ID 999
- **WHEN** the `deleteMember` mutation is executed with id=999
- **THEN** a MemberNotFoundError SHALL be raised
### Requirement: E2E Tests for GraphQL API
End-to-end tests SHALL verify complete GraphQL request/response flows over HTTP.
#### Scenario: Execute GraphQL query via HTTP POST
- **GIVEN** the FastAPI application with GraphQL endpoint
- **WHEN** a POST request is sent to /graphql with a query
- **THEN** the query SHALL be executed
- **AND** a JSON response SHALL be returned with data or errors
#### Scenario: GraphQL introspection query
- **GIVEN** the GraphQL API is running
- **WHEN** an introspection query for __schema is executed
- **THEN** the complete schema SHALL be returned
- **AND** all types, queries, and mutations SHALL be included
#### Scenario: GraphQL type introspection
- **GIVEN** the GraphQL API is running
- **WHEN** an introspection query for __type(name: "Member") is executed
- **THEN** the Member type definition SHALL be returned
- **AND** all fields with their types SHALL be included
### Requirement: MCP Server Tests
Tests SHALL verify MCP server tool definitions and execution.
#### Scenario: MCP server initializes successfully
- **GIVEN** a valid CLUBBER_API_URL
- **WHEN** the MCP server is instantiated
- **THEN** the server SHALL initialize without errors
- **AND** the GraphQL client SHALL be configured
#### Scenario: List tools returns all 6 tools
- **GIVEN** an initialized MCP server
- **WHEN** `list_tools()` is called
- **THEN** 6 tools SHALL be returned
- **AND** tools SHALL include: list_members, get_member, create_member, update_member, get_graphql_schema, execute_graphql_query
#### Scenario: Execute list_members tool
- **GIVEN** members exist in the database
- **WHEN** the `list_members` tool is invoked
- **THEN** a formatted list of all members SHALL be returned
#### Scenario: Execute get_member tool
- **GIVEN** a member exists with ID 1
- **WHEN** the `get_member` tool is invoked with id=1
- **THEN** the member's data SHALL be returned as formatted text
#### Scenario: Execute create_member tool
- **GIVEN** valid member input data
- **WHEN** the `create_member` tool is invoked
- **THEN** a new member SHALL be created via the GraphQL API
- **AND** the created member data SHALL be returned
#### Scenario: Execute get_graphql_schema tool
- **GIVEN** the GraphQL API is accessible
- **WHEN** the `get_graphql_schema` tool is invoked
- **THEN** the complete schema SHALL be retrieved via introspection
- **AND** formatted schema information SHALL be returned
#### Scenario: Execute execute_graphql_query tool with simple query
- **GIVEN** a valid GraphQL query string
- **WHEN** the `execute_graphql_query` tool is invoked
- **THEN** the query SHALL be executed against the API
- **AND** formatted JSON results SHALL be returned
#### Scenario: Execute execute_graphql_query tool with variables
- **GIVEN** a GraphQL query with variables
- **WHEN** the `execute_graphql_query` tool is invoked with query and variables
- **THEN** the query SHALL be executed with variable substitution
- **AND** results SHALL be returned
### Requirement: Test Documentation in README
The README SHALL document how to run tests and understand test organization.
#### Scenario: README contains test running instructions
- **GIVEN** the README.md file
- **WHEN** reading the documentation
- **THEN** a "Running Tests" section SHALL exist
- **AND** the section SHALL document the `uv run pytest` command
- **AND** examples of running specific test categories SHALL be provided
#### Scenario: README documents test organization
- **GIVEN** the README.md file
- **WHEN** reading the documentation
- **THEN** a "Test Organization" section SHALL exist
- **AND** the section SHALL explain the tests/ directory structure
- **AND** the section SHALL describe unit, integration, e2e, and mcp test categories
openspec/changes/add-pytest-tests/tasks.md
+102
View File
@@ -0,0 +1,102 @@
# Implementation Tasks: add-pytest-tests
## 1. Test Infrastructure Setup
- [ ] Create `tests/` directory structure
- [ ] `tests/unit/` for pure logic tests
- [ ] `tests/integration/` for database and resolver tests
- [ ] `tests/e2e/` for full API tests
- [ ] `tests/mcp/` for MCP server tests
- [ ] Create `tests/conftest.py` with shared fixtures
- [ ] Database fixture (in-memory SQLite)
- [ ] Async session fixture
- [ ] Test client fixture for FastAPI/GraphQL
- [ ] Create `tests/__init__.py`
## 2. Unit Tests
- [ ] `tests/unit/test_validation.py`
- [ ] Test `validate_first_name()` with valid input
- [ ] Test `validate_first_name()` with empty string
- [ ] Test `validate_first_name()` with whitespace only
- [ ] Test `validate_email()` with valid emails
- [ ] Test `validate_email()` with invalid emails
- [ ] Test `validate_email()` with None (should pass)
- [ ] Test `validate_email()` with empty string (should pass)
- [ ] Test `validate_phone()` with valid E.164 numbers
- [ ] Test `validate_phone()` with invalid numbers
- [ ] Test `validate_phone()` with None (should pass)
- [ ] Test `validate_phone()` with empty string (should pass)
## 3. Integration Tests - Database
- [ ] `tests/integration/test_member_model.py`
- [ ] Test creating member with minimal data (firstName only)
- [ ] Test creating member with complete data
- [ ] Test member timestamps (created_at, updated_at)
- [ ] Test member __repr__
## 4. Integration Tests - GraphQL Resolvers
- [ ] `tests/integration/test_member_queries.py`
- [ ] Test `member(id)` query with existing member
- [ ] Test `member(id)` query with non-existent ID (returns None)
- [ ] Test `members` query with multiple members
- [ ] Test `members` query returns empty array when no members
- [ ] Test `members` sorting (last name nulls last, then first name)
- [ ] `tests/integration/test_member_mutations.py`
- [ ] Test `createMember` with minimal data (firstName only)
- [ ] Test `createMember` with complete data
- [ ] Test `createMember` with invalid email (validation error)
- [ ] Test `createMember` with invalid phone (validation error)
- [ ] Test `createMember` with empty firstName (validation error)
- [ ] Test `updateMember` successfully updates fields
- [ ] Test `updateMember` with non-existent ID (MemberNotFoundError)
- [ ] Test `updateMember` with partial data (only updates provided fields)
- [ ] Test `updateMember` with invalid email (validation error)
- [ ] Test `deleteMember` successfully deletes member
- [ ] Test `deleteMember` with non-existent ID (MemberNotFoundError)
## 5. E2E Tests - GraphQL API
- [ ] `tests/e2e/test_graphql_api.py`
- [ ] Test complete query flow (list members via HTTP)
- [ ] Test complete mutation flow (create member via HTTP)
- [ ] Test GraphQL introspection query (__schema)
- [ ] Test GraphQL type introspection (__type)
- [ ] Test error responses (400 for validation errors)
## 6. MCP Server Tests
- [ ] `tests/mcp/test_mcp_server.py`
- [ ] Test MCP server initialization
- [ ] Test `list_tools()` returns 6 tools
- [ ] Test `list_members` tool execution
- [ ] Test `get_member` tool execution
- [ ] Test `get_member` with non-existent ID
- [ ] Test `create_member` tool execution
- [ ] Test `update_member` tool execution
- [ ] Test `get_graphql_schema` tool execution
- [ ] Test `execute_graphql_query` tool with simple query
- [ ] Test `execute_graphql_query` tool with variables
- [ ] Test error handling when API is unavailable
## 7. Documentation
- [ ] Update README.md
- [ ] Add "Running Tests" section
- [ ] Add test command examples
- [ ] Add "Test Organization" section
- [ ] Document test coverage goals
## 8. Validation
- [ ] Run `uv run pytest` and verify all tests pass
- [ ] Run `uv run pytest --verbose` for detailed output
- [ ] Run `uv run pytest tests/unit` to test unit tests
- [ ] Run `uv run pytest tests/integration` to test integration tests
- [ ] Run `uv run pytest tests/e2e` to test e2e tests
- [ ] Run `uv run pytest tests/mcp` to test MCP tests
- [ ] Verify no changes to production code (except bug fixes if found)
- [ ] Run `openspec validate add-pytest-tests --strict`