Code Generation Guide¶

Last Updated: 2025-11-23 Difficulty: Intermediate

This guide covers code generation in ocmonica, including protobuf/gRPC code generation, type converters, and documentation generation.

Table of Contents¶

Overview
Protobuf Code Generation
Buf Configuration
Adding New Services
Type Converters
OpenAPI Generation
Frontend Type Generation
Validation
Troubleshooting
Best Practices

Overview¶

What Gets Generated?¶

Ocmonica uses code generation for several purposes:

Protobuf/gRPC Code (Go):
Message types from .proto files
gRPC service interfaces
ConnectRPC handlers
OpenAPI Documentation:
REST API documentation (Swagger)
gRPC API documentation (from protobuf)
Frontend Types (TypeScript):
ConnectRPC client code
Type definitions from protobuf
Status Documentation:
PROJECT_STATUS.md (auto-generated from code analysis)

Generated Code Location¶

ocmonica/
├── gen/                       # Generated Go code (gitignored)
│   └── ocmonica/
│       └── v1/
│           ├── *.pb.go       # Protobuf messages
│           └── *connect.pb.go # ConnectRPC services
├── docs/
│   ├── swagger/              # REST API docs (gitignored)
│   │   ├── docs.go
│   │   ├── swagger.json
│   │   └── swagger.yaml
│   └── grpc/                 # gRPC API docs (gitignored)
│       └── *.swagger.json
└── web/src/gen/              # Frontend types (planned)

Tools Used¶

Buf: Protobuf linting and code generation
protoc-gen-go: Go protobuf compiler
protoc-gen-connect-go: ConnectRPC code generator
swag: Swagger/OpenAPI documentation generator

Protobuf Code Generation¶

Quick Start¶

Generate all protobuf code:

# Full generation (recommended)
task proto:gen

# Or manually with buf
buf generate

What Happens During Generation?¶

Buf reads proto/**/*.proto files
Validates protobuf definitions with linting
Generates Go code:
*.pb.go - Protobuf message types
*connect.pb.go - ConnectRPC service stubs
Generates OpenAPI documentation for gRPC services
Outputs to gen/ directory (gitignored)

File Structure¶

Source (proto/):

proto/
├── buf.yaml                  # Buf configuration
└── ocmonica/
    └── v1/
        ├── file.proto       # File messages and FileService
        ├── directory.proto  # Directory messages and DirectoryService
        ├── search.proto     # Search messages and SearchService
        └── common.proto     # Shared message types

Generated (gen/):

gen/
└── ocmonica/
    └── v1/
        ├── file.pb.go              # File protobuf messages
        ├── fileconnect.pb.go       # FileService ConnectRPC handlers
        ├── directory.pb.go         # Directory messages
        ├── directoryconnect.pb.go  # DirectoryService handlers
        ├── search.pb.go            # Search messages
        ├── searchconnect.pb.go     # SearchService handlers
        └── common.pb.go            # Shared messages

Generation Process¶

# Clean generated code
rm -rf gen/

# Regenerate everything
task proto:gen

# Check what changed
git diff gen/

When to regenerate: - After modifying .proto files - After updating Buf plugins - After pulling changes from upstream - When generated code is missing

Buf Configuration¶

buf.yaml (Linting & Breaking Changes)¶

version: v2
modules:
  - path: proto            # Location of .proto files
lint:
  use:
    - STANDARD             # Standard linting rules
breaking:
  use:
    - FILE                 # Breaking change detection

Linting rules (STANDARD includes): - File naming (snake_case) - Message naming (PascalCase) - Field naming (snake_case) - Service naming (PascalCase) - RPC naming (PascalCase) - Proper imports and dependencies

buf.gen.yaml (Code Generation)¶

version: v2
managed:
  enabled: true
  override:
    - file_option: go_package_prefix
      value: github.com/j-pye/ocmonica/gen

plugins:
  # Go protobuf generation
  - local: protoc-gen-go
    out: gen
    opt:
      - paths=source_relative

  # ConnectRPC generation
  - local: protoc-gen-connect-go
    out: gen
    opt:
      - paths=source_relative

  # OpenAPI v2 (Swagger) for gRPC
  - remote: buf.build/grpc-ecosystem/openapiv2:v2.25.1
    out: docs/grpc
    opt:
      - json_names_for_fields=true
      - generate_unbound_methods=true

Plugin configuration: - local: Uses locally installed plugins - remote: Uses Buf's remote plugins (automatic download) - out: Output directory - opt: Plugin-specific options

Buf Commands¶

# Lint protobuf files
buf lint

# Check for breaking changes
buf breaking --against '.git#branch=main'

# Format protobuf files
buf format -w

# Generate code
buf generate

# Update dependencies
buf mod update

# List all modules
buf dep update

Adding New Services¶

Step 1: Define Protobuf Messages¶

Create or edit a .proto file in proto/ocmonica/v1/:

// proto/ocmonica/v1/notification.proto
syntax = "proto3";

package ocmonica.v1;

import "google/protobuf/timestamp.proto";

option go_package = "ocmonica/v1";

// Notification message
message Notification {
  string id = 1;
  string user_id = 2;
  string title = 3;
  string message = 4;
  bool read = 5;
  google.protobuf.Timestamp created_at = 6;
}

// Request messages
message GetNotificationRequest {
  string id = 1;
}

message ListNotificationsRequest {
  int32 limit = 1;
  int32 offset = 2;
  bool unread_only = 3;
}

message MarkAsReadRequest {
  string id = 1;
}

// Response messages
message GetNotificationResponse {
  Notification notification = 1;
}

message ListNotificationsResponse {
  repeated Notification notifications = 1;
  int32 total = 2;
}

message MarkAsReadResponse {
  Notification notification = 1;
}

Step 2: Define Service¶

Add service definition in the same file or a separate one:

// NotificationService handles user notifications
service NotificationService {
  // Get a single notification by ID
  rpc GetNotification(GetNotificationRequest) returns (GetNotificationResponse);

  // List notifications with pagination
  rpc ListNotifications(ListNotificationsRequest) returns (ListNotificationsResponse);

  // Mark notification as read
  rpc MarkAsRead(MarkAsReadRequest) returns (MarkAsReadResponse);
}

Step 3: Generate Code¶

# Lint first (catch errors early)
task proto:lint

# Generate code
task proto:gen

This creates: - gen/ocmonica/v1/notification.pb.go - Message types - gen/ocmonica/v1/notificationconnect.pb.go - Service interface

Step 4: Implement Service Handler¶

Create internal/api/grpc/notification_service.go:

package grpc

import (
    "context"

    "connectrpc.com/connect"
    v1 "github.com/j-pye/ocmonica/gen/ocmonica/v1"
    "github.com/j-pye/ocmonica/internal/service"
)

// NotificationServiceHandler implements the NotificationService gRPC service.
type NotificationServiceHandler struct {
    notificationService *service.NotificationService
}

// NewNotificationServiceHandler creates a new notification service handler.
func NewNotificationServiceHandler(ns *service.NotificationService) *NotificationServiceHandler {
    return &NotificationServiceHandler{
        notificationService: ns,
    }
}

// GetNotification retrieves a notification by ID.
func (h *NotificationServiceHandler) GetNotification(
    ctx context.Context,
    req *connect.Request[v1.GetNotificationRequest],
) (*connect.Response[v1.GetNotificationResponse], error) {
    // Call service layer
    notification, err := h.notificationService.Get(ctx, req.Msg.Id)
    if err != nil {
        return nil, connect.NewError(connect.CodeInternal, err)
    }

    // Convert to protobuf
    resp := &v1.GetNotificationResponse{
        Notification: converters.NotificationToProto(notification),
    }

    return connect.NewResponse(resp), nil
}

// Implement other RPC methods...

Step 5: Register Service¶

In cmd/server/main.go:

// Create service handler
notificationHandler := grpc.NewNotificationServiceHandler(notificationService)

// Register with ConnectRPC
path, handler := ocmonicav1connect.NewNotificationServiceHandler(notificationHandler)
mux.Handle(path, handler)

Step 6: Write Tests¶

Create internal/api/grpc/notification_service_test.go:

func TestNotificationServiceHandler_GetNotification(t *testing.T) {
    client, cleanup := setupTestNotificationServiceHandler(t)
    defer cleanup()

    ctx := context.Background()

    t.Run("successful retrieval", func(t *testing.T) {
        // Create test notification
        // ...

        // Call gRPC method
        resp, err := client.GetNotification(ctx, &connect.Request[v1.GetNotificationRequest]{
            Msg: &v1.GetNotificationRequest{Id: notificationID},
        })

        // Assertions
        require.NoError(t, err)
        assert.Equal(t, notificationID, resp.Msg.Notification.Id)
    })
}

Type Converters¶

Purpose¶

Convert between domain models and protobuf messages.

Pattern¶

Create converter functions in internal/api/grpc/converters/:

// internal/api/grpc/converters/file.go
package converters

import (
    v1 "github.com/j-pye/ocmonica/gen/ocmonica/v1"
    "github.com/j-pye/ocmonica/internal/models"
    "google.golang.org/protobuf/types/known/timestamppb"
)

// FileToProto converts a domain File to a protobuf File.
func FileToProto(f *models.File) *v1.File {
    if f == nil {
        return nil
    }

    protoFile := &v1.File{
        Id:        f.ID,
        Name:      f.Name,
        Path:      f.Path,
        Size:      f.Size,
        MimeType:  f.MimeType,
        CreatedAt: timestamppb.New(f.CreatedAt),
        UpdatedAt: timestamppb.New(f.UpdatedAt),
    }

    // Handle optional fields
    if f.ParentID != nil {
        protoFile.ParentId = *f.ParentID
    }

    return protoFile
}

// ProtoToFile converts a protobuf File to a domain File.
func ProtoToFile(pf *v1.File) *models.File {
    if pf == nil {
        return nil
    }

    file := &models.File{
        ID:        pf.Id,
        Name:      pf.Name,
        Path:      pf.Path,
        Size:      pf.Size,
        MimeType:  pf.MimeType,
        CreatedAt: pf.CreatedAt.AsTime(),
        UpdatedAt: pf.UpdatedAt.AsTime(),
    }

    // Handle optional fields
    if pf.ParentId != "" {
        file.ParentID = &pf.ParentId
    }

    return file
}

Usage in Handlers¶

// In gRPC handler
func (h *FileServiceHandler) GetFile(
    ctx context.Context,
    req *connect.Request[v1.GetFileRequest],
) (*connect.Response[v1.GetFileResponse], error) {
    // Service layer returns domain model
    file, err := h.fileService.Get(ctx, req.Msg.Id)
    if err != nil {
        return nil, connect.NewError(connect.CodeInternal, err)
    }

    // Convert to protobuf
    resp := &v1.GetFileResponse{
        File: converters.FileToProto(file),
    }

    return connect.NewResponse(resp), nil
}

Best Practices¶

Always handle nil:
```
if f == nil {
    return nil
}
```

Handle optional fields properly:

if f.ParentID != nil {
    protoFile.ParentId = *f.ParentID
}

Convert timestamps correctly:

import "google.golang.org/protobuf/types/known/timestamppb"

CreatedAt: timestamppb.New(f.CreatedAt),  // To protobuf
CreatedAt: pf.CreatedAt.AsTime(),         // From protobuf

Create separate converters for lists:

func FilesToProto(files []*models.File) []*v1.File {
    result := make([]*v1.File, len(files))
    for i, f := range files {
        result[i] = FileToProto(f)
    }
    return result
}

OpenAPI Generation¶

REST API (Swagger)¶

Generated from Go code annotations:

# Generate Swagger docs
task swagger:gen

# Or manually
swag init -g cmd/server/main.go -o docs/swagger

Output: - docs/swagger/docs.go - Embedded documentation - docs/swagger/swagger.json - OpenAPI JSON spec - docs/swagger/swagger.yaml - OpenAPI YAML spec

View documentation:

# Start server
task dev

# Open in browser
open http://localhost:8080/swagger/index.html

gRPC API (from Protobuf)¶

Generated automatically by Buf:

# In buf.gen.yaml
plugins:
  - remote: buf.build/grpc-ecosystem/openapiv2:v2.25.1
    out: docs/grpc
    opt:
      - json_names_for_fields=true
      - generate_unbound_methods=true

Output: - docs/grpc/*.swagger.json - One file per service

Viewing: - Import JSON into Swagger UI - Use Postman/Insomnia - Use gRPC tools like grpcurl

Frontend Type Generation¶

TypeScript from Protobuf (Planned)¶

Future integration with ConnectRPC TypeScript:

# buf.gen.yaml (planned)
plugins:
  - remote: buf.build/connectrpc/es
    out: web/src/gen
    opt:
      - target=ts

Will generate: - TypeScript type definitions - ConnectRPC client code - Type-safe API calls

Current Approach¶

Manually maintain TypeScript types:

// web/src/types/file.ts
export interface File {
  id: string;
  name: string;
  path: string;
  size: number;
  mimeType: string;
  createdAt: string;
  updatedAt: string;
  parentId?: string;
}

Validation¶

Protobuf Linting¶

# Lint all .proto files
task proto:lint

# Or manually
buf lint

# Check specific file
buf lint proto/ocmonica/v1/file.proto

Common linting errors:

Incorrect naming:

proto/ocmonica/v1/file.proto:10:3:Field name "FileID" should be lower_snake_case.

Fix: Use file_id instead of FileID

Missing package:

proto/ocmonica/v1/file.proto:1:1:File does not have a package declaration.

Fix: Add package ocmonica.v1;

Wrong import path:
```
proto/ocmonica/v1/file.proto:3:8:Import "common.proto" not found.
```
Fix: Use relative path import "ocmonica/v1/common.proto";

Breaking Change Detection¶

# Check against main branch
buf breaking --against '.git#branch=main'

# Check against specific commit
buf breaking --against '.git#commit=abc123'

# Check against remote
buf breaking --against 'https://github.com/j-pye/ocmonica.git#branch=main'

Example output:

proto/ocmonica/v1/file.proto:15:3:Field "1" on message "File" changed type from "string" to "int64".

Allowed changes: - Adding new fields - Adding new messages - Adding new services - Adding new RPC methods

Breaking changes: - Changing field types - Changing field numbers - Removing fields - Renaming messages

Code Quality¶

# Format protobuf files
buf format -w

# Lint Go generated code
golangci-lint run ./gen/...

# Verify generation is up-to-date
task proto:gen
git diff --exit-code gen/

Troubleshooting¶

Generation Failures¶

Problem: buf generate fails

Solutions: 1. Check buf is installed: buf --version 2. Check plugins are installed:

which protoc-gen-go
which protoc-gen-connect-go

3. Install missing plugins:

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install connectrpc.com/connect/cmd/protoc-gen-connect-go@latest

4. Clear cache: buf mod clear-cache

Problem: Generated code has errors

Solutions: 1. Run buf lint to catch protobuf errors 2. Regenerate: rm -rf gen/ && buf generate 3. Check imports in .proto files 4. Verify buf.gen.yaml configuration

Import Errors¶

Problem: import "file.proto" not found

Solutions: 1. Use correct import path: import "ocmonica/v1/file.proto" 2. Ensure file exists in proto/ directory 3. Check buf.yaml modules configuration

Problem: Go imports fail after generation

Solutions: 1. Run go mod tidy 2. Check go_package option in .proto files 3. Verify buf.gen.yaml go_package_prefix

Stale Generated Code¶

Problem: Changes to .proto files not reflected

Solutions:

# Clean and regenerate
rm -rf gen/
task proto:gen

# Restart dev server
task dev

Problem: IDE shows old types

Solutions: 1. Restart IDE 2. Clear IDE cache (VS Code: Reload Window) 3. Run go mod tidy

Performance Issues¶

Problem: Generation is slow

Causes: - Large number of .proto files - Complex dependencies - Remote plugins downloading

Solutions: 1. Use local plugins when possible 2. Cache remote plugins: buf mod clear-cache then regenerate 3. Generate only changed services (manual protoc invocation)

Best Practices¶

Protobuf Design¶

Use semantic versioning in package names:

package ocmonica.v1;  // v1, v2, v3, etc.

Never reuse field numbers:

message File {
  string id = 1;
  string name = 2;
  // DO NOT reuse 3 if a field was deleted
  int64 size = 4;
}

Use appropriate field types:

import "google/protobuf/timestamp.proto";

message File {
  google.protobuf.Timestamp created_at = 5;  // Not string
}

Document all messages and fields:

// File represents a stored file in the system.
message File {
  // Unique identifier for the file
  string id = 1;

  // Display name of the file
  string name = 2;
}

Group related messages:
Put service + related messages in same file
Use common.proto for shared types
Keep files focused and cohesive

Code Generation¶

Commit generated code? NO
Generated code is gitignored
Regenerate in CI/CD
Include in .dockerignore
When to regenerate:
After every .proto change
Before committing changes
After pulling changes from upstream
In CI pipeline
Version control:
Commit .proto files
Commit buf.yaml and buf.gen.yaml
Do NOT commit gen/ directory
Document generation steps in README

Compatibility¶

Maintain backward compatibility:
Add fields, don't modify existing ones
Use optional fields for new additions
Create new messages for breaking changes

Test compatibility:

buf breaking --against '.git#branch=main'

Version your API:
Use v1, v2 packages for major versions
Keep old versions available during migration
Document migration paths

Quick Reference¶

Common Tasks¶

# Generate all code
task proto:gen

# Lint protobuf
task proto:lint

# Format protobuf
task proto:fmt

# Check breaking changes
buf breaking --against '.git#branch=main'

# Generate Swagger docs
task swagger:gen

# Clean generated code
rm -rf gen/ docs/swagger/ docs/grpc/

# Full rebuild
rm -rf gen/ && task proto:gen && task swagger:gen

File Checklist¶

When adding a new service, create/update:

Resources¶

Documentation¶

Tools¶

Examples in Project¶

Simple service: proto/ocmonica/v1/search.proto
Complex service: proto/ocmonica/v1/file.proto
Handler implementation: internal/api/grpc/file_service.go
Converters: internal/api/grpc/converters/file.go

Getting Help¶

If you have questions about code generation:

Check Buf documentation
Look at existing .proto files for patterns
Ask in GitHub Discussions
Open an issue for clarification

Happy generating!