diff --git a/openspec/changes/add-pytest-tests/proposal.md b/openspec/changes/add-pytest-tests/proposal.md new file mode 100644 index 0000000..c45b997 --- /dev/null +++ b/openspec/changes/add-pytest-tests/proposal.md @@ -0,0 +1,91 @@ +# Add Pytest Tests Proposal + +## Why + +The project currently has no automated tests despite having pytest and pytest-asyncio configured in `pyproject.toml`. This creates several risks: +- **Regression risk**: Changes to existing features (member CRUD operations, GraphQL resolvers, MCP tools, validation logic) may break functionality without detection +- **Confidence**: Developers and AI assistants lack confidence when refactoring or adding features +- **Documentation gap**: Tests serve as executable documentation of expected behavior +- **Quality assurance**: No automated verification of business rules (validation, sorting, error handling) + +Testing is critical for: +1. Member CRUD operations (create, read, update, delete) +2. GraphQL API (queries, mutations, introspection) +3. MCP server tools (6 tools connecting to GraphQL API) +4. Validation logic (email, phone, firstName) +5. Database operations (async SQLAlchemy patterns) + +## What + +Implement comprehensive pytest test suite covering existing features: + +### Test Infrastructure +- Create `tests/` directory with proper structure (unit, integration, e2e, mcp) +- Configure pytest fixtures for database setup (in-memory SQLite) +- Configure async test support (pytest-asyncio already in dependencies) +- Add conftest.py with shared fixtures + +### Test Coverage +1. **Unit Tests** (`tests/unit/`) + - Validation logic (email, phone, firstName) + - Utility functions + +2. **Integration Tests** (`tests/integration/`) + - Database operations with Member model + - GraphQL resolvers (Query, Mutation) + - Error handling (MemberNotFoundError, ValidationError) + +3. **E2E Tests** (`tests/e2e/`) + - Complete GraphQL query flows + - Complete GraphQL mutation flows + - GraphQL introspection queries + +4. **MCP Tests** (`tests/mcp/`) + - MCP server tool definitions + - MCP tool execution (list_members, get_member, create_member, update_member) + - General GraphQL tools (get_graphql_schema, execute_graphql_query) + +### Documentation +- Update README.md with testing instructions +- Add "Running Tests" section +- Add "Test Coverage" section +- Document test organization and conventions + +## Impact + +### Breaking Changes +None - purely additive. + +### New Dependencies +None - pytest and pytest-asyncio already in dev dependencies. + +### Migration +No migration needed. + +## Alternatives Considered + +1. **unittest instead of pytest** + - Rejected: pytest is already configured and provides better async support, fixtures, and test discovery + +2. **Test only critical paths** + - Rejected: Comprehensive testing provides better confidence and documentation + +3. **Skip MCP tests** + - Rejected: MCP server is a core component and needs testing + +## Success Criteria + +1. Test suite runs with `uv run pytest` +2. All existing features have test coverage +3. Tests are organized logically (unit, integration, e2e, mcp) +4. Tests use proper fixtures for database setup +5. README.md documents how to run tests +6. No changes to production code (except minor fixes if bugs are found) + +## Related Changes + +None - this is the first testing implementation. + +## Open Questions + +None - the implementation is straightforward. diff --git a/openspec/changes/add-pytest-tests/specs/project-setup/spec.md b/openspec/changes/add-pytest-tests/specs/project-setup/spec.md new file mode 100644 index 0000000..71266ea --- /dev/null +++ b/openspec/changes/add-pytest-tests/specs/project-setup/spec.md @@ -0,0 +1,295 @@ +# project-setup Spec Delta + +## ADDED Requirements + +### Requirement: Automated Test Suite with Pytest +The project SHALL provide a comprehensive test suite using pytest to verify all features and prevent regressions. + +#### Scenario: Run test suite successfully +- **GIVEN** the project is set up with dependencies installed +- **WHEN** the developer runs `uv run pytest` +- **THEN** all tests SHALL execute +- **AND** test results SHALL be displayed with pass/fail status +- **AND** the command SHALL exit with code 0 if all tests pass + +#### Scenario: Run tests with verbose output +- **GIVEN** the project has tests implemented +- **WHEN** the developer runs `uv run pytest --verbose` +- **THEN** detailed test output SHALL be displayed +- **AND** each test name and result SHALL be shown + +#### Scenario: Run specific test categories +- **GIVEN** tests are organized in subdirectories +- **WHEN** the developer runs `uv run pytest tests/unit` +- **THEN** only unit tests SHALL execute +- **WHEN** the developer runs `uv run pytest tests/integration` +- **THEN** only integration tests SHALL execute + +### Requirement: Test Organization Structure +Tests SHALL be organized in a clear directory structure separating different test types. + +#### Scenario: Test directory structure exists +- **GIVEN** the project repository +- **WHEN** inspecting the directory structure +- **THEN** a `tests/` directory SHALL exist at the project root +- **AND** subdirectories `tests/unit/`, `tests/integration/`, `tests/e2e/`, and `tests/mcp/` SHALL exist +- **AND** a `tests/conftest.py` file SHALL exist for shared fixtures + +#### Scenario: Test files follow naming convention +- **GIVEN** test files in the `tests/` directory +- **WHEN** examining file names +- **THEN** all test files SHALL be named `test_*.py` +- **AND** test file names SHALL correspond to the modules they test (e.g., `test_validation.py` for `validation.py`) + +### Requirement: Shared Test Fixtures +The test suite SHALL provide reusable fixtures for common test setup via conftest.py. + +#### Scenario: Database fixture provides in-memory SQLite +- **GIVEN** tests need database access +- **WHEN** a test function requests the `db` fixture +- **THEN** an in-memory SQLite database SHALL be created +- **AND** the database SHALL have all tables created via Alembic migrations or direct schema creation +- **AND** the database SHALL be cleaned up after the test completes + +#### Scenario: Async session fixture provides database sessions +- **GIVEN** tests need to interact with the database +- **WHEN** a test function requests the `async_session` fixture +- **THEN** an async SQLAlchemy session SHALL be provided +- **AND** the session SHALL be connected to the test database +- **AND** the session SHALL be closed after the test completes + +#### Scenario: GraphQL client fixture provides test client +- **GIVEN** tests need to make GraphQL requests +- **WHEN** a test function requests the `graphql_client` fixture +- **THEN** a FastAPI test client SHALL be provided +- **AND** the client SHALL be configured to use the test database +- **AND** the client SHALL support async GraphQL queries and mutations + +### Requirement: Unit Tests for Validation Logic +Unit tests SHALL verify validation functions work correctly with valid and invalid inputs. + +#### Scenario: Test firstName validation with valid input +- **GIVEN** a valid firstName "Alice" +- **WHEN** `validate_first_name("Alice")` is called +- **THEN** no exception SHALL be raised + +#### Scenario: Test firstName validation with empty string +- **GIVEN** an empty firstName "" +- **WHEN** `validate_first_name("")` is called +- **THEN** a ValidationError SHALL be raised +- **AND** the error message SHALL indicate firstName cannot be empty + +#### Scenario: Test email validation with valid email +- **GIVEN** a valid email "test@example.com" +- **WHEN** `validate_email("test@example.com")` is called +- **THEN** no exception SHALL be raised + +#### Scenario: Test email validation with invalid email +- **GIVEN** an invalid email "not-an-email" +- **WHEN** `validate_email("not-an-email")` is called +- **THEN** a ValidationError SHALL be raised +- **AND** the error message SHALL indicate invalid email format + +#### Scenario: Test email validation with None +- **GIVEN** email is None +- **WHEN** `validate_email(None)` is called +- **THEN** no exception SHALL be raised (None is allowed) + +#### Scenario: Test phone validation with valid E.164 number +- **GIVEN** a valid phone "+41791234567" +- **WHEN** `validate_phone("+41791234567")` is called +- **THEN** no exception SHALL be raised + +#### Scenario: Test phone validation with invalid number +- **GIVEN** an invalid phone "123" +- **WHEN** `validate_phone("123")` is called +- **THEN** a ValidationError SHALL be raised + +#### Scenario: Test phone validation with None +- **GIVEN** phone is None +- **WHEN** `validate_phone(None)` is called +- **THEN** no exception SHALL be raised (None is allowed) + +### Requirement: Integration Tests for Member Model +Integration tests SHALL verify database operations with the Member model. + +#### Scenario: Create member with minimal data +- **GIVEN** a test database session +- **WHEN** a Member is created with only firstName="Alice" +- **THEN** the member SHALL be persisted to the database +- **AND** the member SHALL have an auto-generated ID +- **AND** created_at and updated_at timestamps SHALL be populated +- **AND** all optional fields SHALL be None + +#### Scenario: Create member with complete data +- **GIVEN** a test database session +- **WHEN** a Member is created with all fields populated +- **THEN** the member SHALL be persisted with all data +- **AND** all fields SHALL match the provided values + +#### Scenario: Member timestamps are set correctly +- **GIVEN** a newly created member +- **THEN** created_at SHALL be set to current time +- **AND** updated_at SHALL be set to current time +- **AND** created_at SHALL equal updated_at for new members + +### Requirement: Integration Tests for GraphQL Queries +Integration tests SHALL verify GraphQL query resolvers return correct data. + +#### Scenario: Query single member by ID +- **GIVEN** a member exists with ID 1 +- **WHEN** the `member(id: 1)` query is executed +- **THEN** the member data SHALL be returned +- **AND** all requested fields SHALL match the database record + +#### Scenario: Query member with non-existent ID +- **GIVEN** no member exists with ID 999 +- **WHEN** the `member(id: 999)` query is executed +- **THEN** None SHALL be returned +- **AND** no error SHALL be raised + +#### Scenario: List all members +- **GIVEN** multiple members exist in the database +- **WHEN** the `members` query is executed +- **THEN** an array of all members SHALL be returned +- **AND** members SHALL be sorted by last_name (nulls last), then first_name + +#### Scenario: List members when database is empty +- **GIVEN** no members exist in the database +- **WHEN** the `members` query is executed +- **THEN** an empty array SHALL be returned + +### Requirement: Integration Tests for GraphQL Mutations +Integration tests SHALL verify GraphQL mutation resolvers create, update, and delete members correctly. + +#### Scenario: Create member with minimal data +- **GIVEN** CreateMemberInput with only firstName="Bob" +- **WHEN** the `createMember` mutation is executed +- **THEN** a new member SHALL be created in the database +- **AND** the member SHALL have firstName="Bob" +- **AND** all optional fields SHALL be None +- **AND** the mutation SHALL return the created member with ID + +#### Scenario: Create member with complete data +- **GIVEN** CreateMemberInput with all fields populated +- **WHEN** the `createMember` mutation is executed +- **THEN** a new member SHALL be created with all data +- **AND** all fields SHALL match the input + +#### Scenario: Create member with invalid email +- **GIVEN** CreateMemberInput with email="not-an-email" +- **WHEN** the `createMember` mutation is executed +- **THEN** a ValidationError SHALL be raised +- **AND** no database record SHALL be created + +#### Scenario: Update member successfully +- **GIVEN** a member exists with ID 1 +- **WHEN** the `updateMember` mutation is executed with id=1 and email="new@example.com" +- **THEN** the member's email SHALL be updated to "new@example.com" +- **AND** the updated_at timestamp SHALL be refreshed +- **AND** all other fields SHALL remain unchanged + +#### Scenario: Update member with non-existent ID +- **GIVEN** no member exists with ID 999 +- **WHEN** the `updateMember` mutation is executed with id=999 +- **THEN** a MemberNotFoundError SHALL be raised +- **AND** the error message SHALL indicate member not found + +#### Scenario: Delete member successfully +- **GIVEN** a member exists with ID 1 +- **WHEN** the `deleteMember` mutation is executed with id=1 +- **THEN** the member SHALL be removed from the database +- **AND** the mutation SHALL return true +- **AND** subsequent queries for that member SHALL return None + +#### Scenario: Delete member with non-existent ID +- **GIVEN** no member exists with ID 999 +- **WHEN** the `deleteMember` mutation is executed with id=999 +- **THEN** a MemberNotFoundError SHALL be raised + +### Requirement: E2E Tests for GraphQL API +End-to-end tests SHALL verify complete GraphQL request/response flows over HTTP. + +#### Scenario: Execute GraphQL query via HTTP POST +- **GIVEN** the FastAPI application with GraphQL endpoint +- **WHEN** a POST request is sent to /graphql with a query +- **THEN** the query SHALL be executed +- **AND** a JSON response SHALL be returned with data or errors + +#### Scenario: GraphQL introspection query +- **GIVEN** the GraphQL API is running +- **WHEN** an introspection query for __schema is executed +- **THEN** the complete schema SHALL be returned +- **AND** all types, queries, and mutations SHALL be included + +#### Scenario: GraphQL type introspection +- **GIVEN** the GraphQL API is running +- **WHEN** an introspection query for __type(name: "Member") is executed +- **THEN** the Member type definition SHALL be returned +- **AND** all fields with their types SHALL be included + +### Requirement: MCP Server Tests +Tests SHALL verify MCP server tool definitions and execution. + +#### Scenario: MCP server initializes successfully +- **GIVEN** a valid CLUBBER_API_URL +- **WHEN** the MCP server is instantiated +- **THEN** the server SHALL initialize without errors +- **AND** the GraphQL client SHALL be configured + +#### Scenario: List tools returns all 6 tools +- **GIVEN** an initialized MCP server +- **WHEN** `list_tools()` is called +- **THEN** 6 tools SHALL be returned +- **AND** tools SHALL include: list_members, get_member, create_member, update_member, get_graphql_schema, execute_graphql_query + +#### Scenario: Execute list_members tool +- **GIVEN** members exist in the database +- **WHEN** the `list_members` tool is invoked +- **THEN** a formatted list of all members SHALL be returned + +#### Scenario: Execute get_member tool +- **GIVEN** a member exists with ID 1 +- **WHEN** the `get_member` tool is invoked with id=1 +- **THEN** the member's data SHALL be returned as formatted text + +#### Scenario: Execute create_member tool +- **GIVEN** valid member input data +- **WHEN** the `create_member` tool is invoked +- **THEN** a new member SHALL be created via the GraphQL API +- **AND** the created member data SHALL be returned + +#### Scenario: Execute get_graphql_schema tool +- **GIVEN** the GraphQL API is accessible +- **WHEN** the `get_graphql_schema` tool is invoked +- **THEN** the complete schema SHALL be retrieved via introspection +- **AND** formatted schema information SHALL be returned + +#### Scenario: Execute execute_graphql_query tool with simple query +- **GIVEN** a valid GraphQL query string +- **WHEN** the `execute_graphql_query` tool is invoked +- **THEN** the query SHALL be executed against the API +- **AND** formatted JSON results SHALL be returned + +#### Scenario: Execute execute_graphql_query tool with variables +- **GIVEN** a GraphQL query with variables +- **WHEN** the `execute_graphql_query` tool is invoked with query and variables +- **THEN** the query SHALL be executed with variable substitution +- **AND** results SHALL be returned + +### Requirement: Test Documentation in README +The README SHALL document how to run tests and understand test organization. + +#### Scenario: README contains test running instructions +- **GIVEN** the README.md file +- **WHEN** reading the documentation +- **THEN** a "Running Tests" section SHALL exist +- **AND** the section SHALL document the `uv run pytest` command +- **AND** examples of running specific test categories SHALL be provided + +#### Scenario: README documents test organization +- **GIVEN** the README.md file +- **WHEN** reading the documentation +- **THEN** a "Test Organization" section SHALL exist +- **AND** the section SHALL explain the tests/ directory structure +- **AND** the section SHALL describe unit, integration, e2e, and mcp test categories diff --git a/openspec/changes/add-pytest-tests/tasks.md b/openspec/changes/add-pytest-tests/tasks.md new file mode 100644 index 0000000..2b7ce19 --- /dev/null +++ b/openspec/changes/add-pytest-tests/tasks.md @@ -0,0 +1,102 @@ +# Implementation Tasks: add-pytest-tests + +## 1. Test Infrastructure Setup + +- [ ] Create `tests/` directory structure + - [ ] `tests/unit/` for pure logic tests + - [ ] `tests/integration/` for database and resolver tests + - [ ] `tests/e2e/` for full API tests + - [ ] `tests/mcp/` for MCP server tests +- [ ] Create `tests/conftest.py` with shared fixtures + - [ ] Database fixture (in-memory SQLite) + - [ ] Async session fixture + - [ ] Test client fixture for FastAPI/GraphQL +- [ ] Create `tests/__init__.py` + +## 2. Unit Tests + +- [ ] `tests/unit/test_validation.py` + - [ ] Test `validate_first_name()` with valid input + - [ ] Test `validate_first_name()` with empty string + - [ ] Test `validate_first_name()` with whitespace only + - [ ] Test `validate_email()` with valid emails + - [ ] Test `validate_email()` with invalid emails + - [ ] Test `validate_email()` with None (should pass) + - [ ] Test `validate_email()` with empty string (should pass) + - [ ] Test `validate_phone()` with valid E.164 numbers + - [ ] Test `validate_phone()` with invalid numbers + - [ ] Test `validate_phone()` with None (should pass) + - [ ] Test `validate_phone()` with empty string (should pass) + +## 3. Integration Tests - Database + +- [ ] `tests/integration/test_member_model.py` + - [ ] Test creating member with minimal data (firstName only) + - [ ] Test creating member with complete data + - [ ] Test member timestamps (created_at, updated_at) + - [ ] Test member __repr__ + +## 4. Integration Tests - GraphQL Resolvers + +- [ ] `tests/integration/test_member_queries.py` + - [ ] Test `member(id)` query with existing member + - [ ] Test `member(id)` query with non-existent ID (returns None) + - [ ] Test `members` query with multiple members + - [ ] Test `members` query returns empty array when no members + - [ ] Test `members` sorting (last name nulls last, then first name) + +- [ ] `tests/integration/test_member_mutations.py` + - [ ] Test `createMember` with minimal data (firstName only) + - [ ] Test `createMember` with complete data + - [ ] Test `createMember` with invalid email (validation error) + - [ ] Test `createMember` with invalid phone (validation error) + - [ ] Test `createMember` with empty firstName (validation error) + - [ ] Test `updateMember` successfully updates fields + - [ ] Test `updateMember` with non-existent ID (MemberNotFoundError) + - [ ] Test `updateMember` with partial data (only updates provided fields) + - [ ] Test `updateMember` with invalid email (validation error) + - [ ] Test `deleteMember` successfully deletes member + - [ ] Test `deleteMember` with non-existent ID (MemberNotFoundError) + +## 5. E2E Tests - GraphQL API + +- [ ] `tests/e2e/test_graphql_api.py` + - [ ] Test complete query flow (list members via HTTP) + - [ ] Test complete mutation flow (create member via HTTP) + - [ ] Test GraphQL introspection query (__schema) + - [ ] Test GraphQL type introspection (__type) + - [ ] Test error responses (400 for validation errors) + +## 6. MCP Server Tests + +- [ ] `tests/mcp/test_mcp_server.py` + - [ ] Test MCP server initialization + - [ ] Test `list_tools()` returns 6 tools + - [ ] Test `list_members` tool execution + - [ ] Test `get_member` tool execution + - [ ] Test `get_member` with non-existent ID + - [ ] Test `create_member` tool execution + - [ ] Test `update_member` tool execution + - [ ] Test `get_graphql_schema` tool execution + - [ ] Test `execute_graphql_query` tool with simple query + - [ ] Test `execute_graphql_query` tool with variables + - [ ] Test error handling when API is unavailable + +## 7. Documentation + +- [ ] Update README.md + - [ ] Add "Running Tests" section + - [ ] Add test command examples + - [ ] Add "Test Organization" section + - [ ] Document test coverage goals + +## 8. Validation + +- [ ] Run `uv run pytest` and verify all tests pass +- [ ] Run `uv run pytest --verbose` for detailed output +- [ ] Run `uv run pytest tests/unit` to test unit tests +- [ ] Run `uv run pytest tests/integration` to test integration tests +- [ ] Run `uv run pytest tests/e2e` to test e2e tests +- [ ] Run `uv run pytest tests/mcp` to test MCP tests +- [ ] Verify no changes to production code (except bug fixes if found) +- [ ] Run `openspec validate add-pytest-tests --strict`