Metadata-Version: 2.1
Name: pycftools
Version: 1.0.0
Summary: Additional tools for python-plugins in Cellframe ecosystem
Author-email: Demlabs <support@demlabs.net>
License: MIT
Project-URL: Homepage, https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools
Project-URL: Repository, https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools.git
Project-URL: Documentation, https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools/-/blob/main/README.md
Project-URL: Bug Tracker, https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools/-/issues
Keywords: cellframe,blockchain,database,orm,serialization,web3
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Database
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: System :: Networking :: Monitoring
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic <2.0.0,>=1.10.0
Requires-Dist: SQLAlchemy <2.0.0,>=1.4.0
Requires-Dist: web3 <7.0.0,>=5.0.0
Requires-Dist: eth-typing <5.0.0,>=2.0.0
Requires-Dist: psycopg2-binary >=2.8.0
Provides-Extra: dev
Requires-Dist: pytest >=7.0.0 ; extra == 'dev'
Requires-Dist: pytest-cov >=4.0.0 ; extra == 'dev'
Requires-Dist: black >=23.0.0 ; extra == 'dev'
Requires-Dist: isort >=5.12.0 ; extra == 'dev'
Requires-Dist: flake8 >=6.0.0 ; extra == 'dev'
Requires-Dist: mypy >=1.0.0 ; extra == 'dev'

# PyCFTools

**Additional tools for python-plugins in Cellframe ecosystem**

[![Python 3.7+](https://img.shields.io/badge/python-3.7+-blue.svg)](https://www.python.org/downloads/release/python-370/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Build Status](https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools/badges/main/pipeline.svg)](https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools/-/pipelines)

## Overview

PyCFTools is a comprehensive Python library that provides essential tools for developing applications within the Cellframe blockchain ecosystem. The library offers database ORM models, serialization/deserialization schemas, Web3 contract interaction utilities, and type definitions optimized for Cellframe development.

## Architecture

The library is organized into four main modules:

- **`database`** - SQLAlchemy ORM models for PostgreSQL integration
- **`schemas`** - Pydantic-based serializers/deserializers for Cellframe objects
- **`web3`** - Web3 contract interaction utilities and event scanners
- **`_types`** - Core type definitions and enumerations

## Installation

### From Package Index

```bash
pip install pycftools
```

### From Source

```bash
git clone https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools.git
cd pycftools
pip install -e .
```

### Development Dependencies

```bash
pip install -e ".[dev]"
```

## Quick Start

### Basic Import

```python
import pycftools

# Access version and metadata
print(f"Version: {pycftools.__version__}")
print(f"Available modules: {pycftools.__all__}")
```

### Database Configuration

```python
import pycftools.config as config

# Database connection parameters (configurable via environment variables)
db_url = config.get_database_url()
print(f"Database URL: {db_url}")

# Available environment variables:
# POSTGRES_USER (default: 'postgres')
# POSTGRES_PASSWORD (default: 'postgres')
# DB_HOST (default: 'localhost')
# DB_PORT (default: '5432')
# POSTGRES_DB (default: 'cellframe_db')
```

### Working with Database Models

```python
from pycftools.database import Base, get_engine, get_session
from pycftools.database.models import Net, Chain, Block, Datum, Event

# Initialize database
engine = get_engine()
Session = get_session()

# Create tables
Base.metadata.create_all(engine)

# Work with models
with Session() as session:
    # Create a network
    net = Net(name="testnet")
    session.add(net)
    session.commit()
    
    # Query chains
    chains = session.query(Chain).filter_by(net_id=net.id).all()
```

### Using Type Definitions

```python
from pycftools._types import ChainTypes, DatumTypes, ItemTypes

# Check chain types
if ChainTypes.has_value("esbocs"):
    print("ESBOCS chain type is supported")

# Work with datum types
datum_type = DatumTypes.DATUM_TX
print(f"Transaction datum type: {datum_type}")

# Item types for transactions
item_type = ItemTypes.TX_ITEM_TYPE_IN
```

### Schema Serialization/Deserialization

```python
from pycftools.schemas import Datum, Block, Event, Chain, Net

# Serialize/deserialize Cellframe objects
datum_data = {
    "hash": "abc123...",
    "type": "DATUM_TX",
    "version": "1.0",
    "size": 1024,
    "created_at": "2024-01-01T00:00:00Z",
    "sub_datum": {...}
}

datum = Datum.model_validate(datum_data)
print(f"Datum hash: {datum.hash}")
```

### Web3 Contract Integration

```python
from pycftools.web3.contract import ContractProvider, ContractEventScanner

# Initialize contract provider
provider = ContractProvider(
    name="MyContract",
    provider_urls=["https://rpc.example.com"],
    contract_address="0x1234...",
    abi='[{"type":"function",...}]'
)

# Scan for events
scanner = ContractEventScanner(
    contract_provider=provider,
    event_names=["Transfer", "Approval"],
    state=scanner_state
)

# Get new events
events = scanner.new_events()
```

## Database Models

### Core Models

- **`Net`** - Network definitions
- **`Chain`** - Blockchain chain information  
- **`Block`** - Block data (for ESBOCS chains)
- **`Event`** - Event data (for DAG-PoA chains)
- **`Datum`** - Transaction and data objects
- **`Wallet`** - Wallet addresses
- **`TokenBalance`** - Token balance information

### Model Relationships

```python
# Networks contain multiple chains
net.chains -> List[Chain]

# Chains contain blocks/events and datums
chain.blocks -> List[Block]      # For ESBOCS chains
chain.events -> List[Event]      # For DAG-PoA chains  
chain.datums -> List[Datum]

# Many-to-many relationships
block.datums -> List[Datum]      # Blocks contain datums
event.datums -> List[Datum]      # Events contain datums
```

### Automatic ID Generation

Models automatically generate composite IDs:
- Block ID: `{hash}{chain_id}`
- Event ID: `{hash}{chain_id}`  
- Datum ID: `{hash}{chain_id}`
- Wallet ID: `{addr}{net_id}`

## Schema System

### Available Serializers

- **Net, Chain** - Network and chain serialization
- **Block, Event** - Blockchain structure serialization
- **Datum** - Complex datum serialization with sub-types:
  - `DatumTX` - Transaction data
  - `DatumToken` - Token declarations
  - `DatumEmission` - Token emissions
  - `DatumDecree` - Governance decrees
  - `DatumAnchor` - Anchor transactions
  - `DatumCustom` - Custom data
- **Sign** - Digital signature handling

### Transaction Items

Support for all Cellframe transaction item types:
- `TxIn` - Transaction inputs
- `TxOut` - Transaction outputs  
- `TxOutCond` - Conditional outputs
- `TxSig` - Signatures
- `TxToken` - Token operations
- `TxOutExt` - Extended outputs

## Web3 Integration

### ContractProvider Features

- **Multi-URL Support** - Automatic failover between RPC endpoints
- **POA Middleware** - Built-in Proof of Authority support
- **Event Scanning** - Efficient event log retrieval
- **Transaction Execution** - Smart contract function calls

### Event Scanner

```python
# Initialize scanner state
state = ContractEventScannerState.load(
    provider=provider,
    window=100,  # Block window size
    reader=state_reader_func,  # Optional state persistence
    writer=state_writer_func   # Optional state persistence
)

# Continuous event monitoring
while True:
    new_events = scanner.new_events()
    process_events(new_events)
    time.sleep(30)
```

## Configuration Management

### Environment Variables

```bash
# Database configuration
export POSTGRES_USER=myuser
export POSTGRES_PASSWORD=mypassword
export DB_HOST=localhost
export DB_PORT=5432
export POSTGRES_DB=cellframe_db

# Additional database settings
export DB_ECHO=true              # SQL query logging
export DB_POOL_SIZE=5            # Connection pool size
export DB_POOL_RECYCLE=3600      # Connection recycle time
```

### Backward Compatibility

The library maintains backward compatibility:

```python
# Old style (still supported)
import config
db_url = config.get_database_url()

# New style (recommended)
import pycftools.config as config
db_url = config.get_database_url()
```

## Development

### Prerequisites

- Python 3.7+
- PostgreSQL 9.6+
- Git

### Setup Development Environment

```bash
# Clone repository
git clone https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools.git
cd pycftools

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Linux/Mac
# venv\Scripts\activate   # Windows

# Install development dependencies
pip install -e ".[dev]"

# Install pre-commit hooks (optional)
pre-commit install
```

### Running Tests

```bash
# Run all tests
pytest

# Run with coverage
pytest --cov=pycftools --cov-report=html

# Run specific test file
pytest tests/test_database.py
```

### Code Quality

```bash
# Format code
black pycftools/
isort pycftools/

# Lint code
flake8 pycftools/
mypy pycftools/
```

### Building Package

```bash
# Build wheel package
python -m build --wheel --outdir dist/

# Install locally
pip install dist/pycftools-*.whl
```

## CI/CD Pipeline

The project includes a comprehensive GitLab CI/CD pipeline:

- **Build Stage** - Install dependencies and verify imports
- **Test Stage** - Run comprehensive test suite
- **Package Stage** - Build and test wheel packages
- **Verify Stage** - Final verification from deployed package

### Pipeline Features

- Automated testing across Python versions
- Package integrity verification
- Automatic deployment to internal package repository
- Backward compatibility testing

## API Reference

### Database Module

```python
from pycftools.database import Base, get_engine, get_session
from pycftools.database.models import *
```

### Schema Module

```python
from pycftools.schemas import *
# Imports: Net, Chain, Block, Event, Datum, Sign schemas
```

### Web3 Module  

```python
from pycftools.web3.contract import ContractProvider, ContractEventScanner
```

### Types Module

```python
from pycftools._types import ChainTypes, DatumTypes, ItemTypes
```

## Troubleshooting

### Common Issues

**Import Errors**
```python
# Ensure all dependencies are installed
pip install -r requirements.txt

# Check Python path
import sys
print(sys.path)
```

**Database Connection Issues**
```python
# Verify PostgreSQL is running
# Check environment variables
import os
print(os.getenv('DB_HOST', 'Not set'))

# Test connection
from pycftools.database import get_engine
engine = get_engine()
print(engine.execute('SELECT 1').scalar())
```

**Web3 Connection Issues**
```python
# Verify RPC endpoints are accessible
from pycftools.web3.contract import ContractProvider

provider = ContractProvider(...)
print(f"Connected: {provider.w3.is_connected()}")
```

## Contributing

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Merge Request

### Code Style

- Use Black for code formatting
- Follow PEP 8 guidelines
- Add type hints where appropriate
- Write comprehensive docstrings
- All comments should be in English

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Support

- **Documentation**: [GitLab Repository](https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools)
- **Issues**: [Bug Tracker](https://gitlab.demlabs.net/cellframe/python-cellframe-modules/pycftools/-/issues)
- **Email**: support@demlabs.net

## Changelog

### Version 1.0.0
- Initial release
- Database ORM models
- Schema serialization system
- Web3 contract integration
- Type definitions
- Backward compatibility support
