# Agent Hub

Agent Hub is the central portal for organizations to distribute enterprise-level agents, manage access, and empower users to build their own custom agents.
This is the plugin's repo.

## Template Content

- **.github/**: Contains backend and frontend CI workflows.
- **.vscode/**: Contains settings for linting and launching the webapp from VSCode.
- **backend/**: Contains server files.
- **code-env/**: Contains backend requirements.
- **resource/**: Contains the frontend.
- **tests/**: Contains unit tests.
- **Makefile**: Contains commands to run static checks, linting, and unit tests.
- **pyproject.toml**: Contains formatting, linting, and type-checking rules for Black, Ruff, and Mypy.

## .env

Copy `.env.example` to `.env`

## Installation with uv

1. Create a virtual environment with `uv`:
   ```sh
   uv venv --python <version>
   ```
2. Activate the virtual environment:
   ```sh
   source .venv/bin/activate
   ```
3. Install the dependencies:
   ```sh
   uv pip install -r code-env/python/spec/requirements.dev.txt
   ```
4. (If not done, Copy `.env.example` to `.env`) In`.env` update the values:

   ```
   LOCAL_DEV="true"
   DKU_CURRENT_PROJECT_KEY="your_project_key"
   VITE_API_PORT="5000"
   VITE_CLIENT_PORT="4200"
   VITE_USER_PROFILE="READER"
   ```

5. Create symbolic links to `dataiku` and `dataikuapi` packages from your local DSS_INSTANCE:

   ```sh
   cd your_venv/lib/<PYTHON_VERSION>/site-packages
   ln -s ~/Library/DataScienceStudio/kits/dataiku-dss-<DSS_VERSION>-osx/python/dataikuapi .
   ln -s ~/Library/DataScienceStudio/kits/dataiku-dss-<DSS_VERSION>-osx/python/dataiku .
   ```

   Make sure to replace by your PYTHON and DSS versions .

6. `cd` into `python-lib` and run the backend

```sh
    cd python-lib
    python -m backend.wsgi
```

## Local Backend Development

Useful backend checks:

```sh
make backend-format
make backend-format-check
make backend-lint-check
make backend-lint-fix
make backend-build-check
make backend-unit-tests
make backend-mypy
```

Useful frontend checks:

```sh
make frontend-format-check
make frontend-test-check
make frontend-build-check
```

For frontend-specific setup, tooling, and E2E details, see the dedicated frontend README: resource/frontend/README.md

Install the repository-managed pre-commit hooks with:

```sh
make install-git-hooks
```

The hooks check backend Black formatting and frontend Prettier formatting before commit.

For local development, you need to apply the following changes:

1. Comment out `with dataiku.WebappImpersonationContext() as context:` in

- `python-lib/backend/utils/dss_utils.py`

Like so 👇

```python
# with dataiku.WebappImpersonationContext() as context:
```

2. In `python-lib/backend/utils/ws_utils.py` replace

- ~~`g.authIdentifier = None`~~
- `g.authIdentifier = "admin"`

```python
g.authIdentifier = "admin"
```

3. In `user_utils.py` comment out :

```
# Only send callerTicket for dss >= 14.3.2 to make sure they are redacted properly
    dss_version = client.get_instance_info().raw.get("dssVersion", "0.0.0")
    if (compare_versions(dss_version, "14.3.2")) >= 0:
        cont["dkuCallerTicket"] = client.get_ticket_from_browser_headers(headers_dict=dict(request.headers))
```

4. To use the semantic model features locally, in `resource/frontend/src/lib/semanticModelUtils.ts` replace:

- ~~`let compatible_features: SemanticModelFeature[] = []`~~
- `let compatible_features: SemanticModelFeature[] = [DATASET_SQL_INTEGRATION, END_USER_SQL_EXECUTION_OVERRIDE]`

### Dev Mode

This setup lets you keep local-only changes (like debug settings) without committing them by mistake.

---

### Create the patch

```bash
git diff > dev.patch
```

### Enable dev mode

```bash
make dev-on
```

### Disable dev mode

```bash
make dev-off
```

### Commit protection

If dev.patch is still applied, commits will be blocked with: ⚠️ dev.patch is applied. Run: make dev-off

dev.patch is ignored by Git.

## Database Migrations with Alembic

This project uses Alembic to manage database schema migrations.

### 1. Environment Setup

Before running any Alembic commands, you need to create a `.env` file in the root of the project. This file should contain the following:

```
DATABASE_URL=your_database_url
```

Replace `your_database_url` with the connection string for your database. For example:

```
DATABASE_URL=sqlite:///data_store.db
```

- Run

```
export PYTHONPATH=$(pwd)/python-lib
```

### 2. Generating a New Migration

When you make changes to your SQLAlchemy models, you need to generate a new migration script. This script will contain the necessary code to update the database schema.

To generate a new migration, run the following command:

```bash
.venv/bin/alembic revision --autogenerate -m "Your migration message"
```

Replace `"Your migration message"` with a short, descriptive message about the changes you made.

This will create a new file in the `alembic/versions` directory. Review and adapt as needed.

### 3. Applying a Migration

To apply the latest migration to your database, run the following command:

```bash
.venv/bin/alembic upgrade head
```

This will execute the migration scripts and update your database schema to match your models.

### 4. Managing an Existing Database

If you are integrating Alembic into a project with an existing database, you should not run `alembic upgrade head` directly. This will result in errors because Alembic will try to create tables that already exist.

Instead, you need to "stamp" the database with the latest revision. This tells Alembic that the database is already at the latest version, without running any migrations.

To stamp your database, run the following command:

```bash
PYTHONPATH=python-lib .venv/bin/alembic stamp head
```

After stamping, you can generate and apply new migrations as described in the sections above.

### 5. Production & Team Workflow

The `alembic/versions` directory is the source of truth for your database schema. To ensure consistency across all environments (dev, production, etc.), you should:

1.  **Commit Migration Scripts:** Always commit the migration scripts generated in the `alembic/versions` directory.
2.  **Run Migrations on Deploy:** Include the `alembic upgrade head` command in your deployment process. This will ensure that the production database is always up-to-date with the latest schema.
3.  **Staying in Sync:** You should run `alembic upgrade head` after pulling the latest changes to update your local databases.


## Playwright tests

### Overview
The playwright tests replicate Dataiku isolation setups with isolated permissions between 'Agent Hub administrators' and 'Agent Hub consumers':
- Tests are run parallelly. For this, pools of 5 users are created and dispatched in Playwright tests.
- 'Agent Hub administrators' (Users 'agent_hub_admin_1, ...,'agent_hub_admin_5') are users that run the webapp backends and have admin privileges on the Agent Hub webapp.
   - They can impersonate the 'Agent Hub consumers' in webapps.
- 'Agent Hub consumers' (Users 'agent_hub_consumer_1', ..., 'agent_hub_consumer_5') are users that only consume the Agent Hub webapp.
   - They can't access the admin settings
   - They can access resources on the instance (projects, workspaces) that the 'Agent Hub administrators' can't access.
- The setup can be automatically done locally by running makefile commands from 'playwright.mk':
      - `make  setup-playwright-groups-and-users`: creates the admin/consumers groups and users involved in the tests.
      - `make setup-playwright-projects`: downloads all the test-integration projects and creates local dependencies used in the tests.
         - ⚠️ The Instance projects folder to store 'My Agents' projects is created by the command but needs to be re-selected manually in Agent Hub webapps after downloading the 'AH_PLAYWRIGHT' project.
      -**NOTE** ⚠️ Pay attention to the instance that your target with the dataiku api client: for this, refer to your `~/.dataiku/config.json`
- Each test needs to precise which type of user it acts on, using the function `createAgentHubTest`. Examples: 

```
const { expect, test } = createAgentHubTest('admin')
const { expect, test } = createAgentHubTest('consumer')
```

### Requirements
Running the playwright tests locally requires: 
- An OpenAI connection explicitely named **bs-openai**
- A writable local PostgreSQL connection (Ieally, named 'local_dku_pg').
- To create, from test-integration, a Global API key for the 'administrators' group (https://doc.dataiku.com/dss/latest/publicapi/keys.html#global-api-keys).
    - Keep this key to replace the parameter `TEST_INTEGRATION_SECRET_KEY` in your `.env`


### Setup
(If not done, Copy `.env.example` to `.env`) In`.env` update the value:
```
TEST_INTEGRATION_SECRET_KEY="dkuaps-XYZ"
```

+ Update, if needed: 
```
LOCAL_DKU_PG_CONN_NAME="local_dku_pg"
## Playwright E2E Agent Hub URLs
E2E_BASE_URL="http://localhost:8082/webapps/AH/JK5X3iF/"
E2E_BASE_URL_CONTAINER="http://localhost:8082/webapps/AH/JK5X3iF-container/"
E2E_LIGHT_BASE_URL="http://localhost:8082/webapps/AH/RMIA11D/"
```

The Playwright-only variables are grouped in `.env.example` under the `Playwright E2E` section:

- `E2E_BASE_URL` - sqlite/default-storage agent-hub webapp url used when `E2E_AGENTHUB_TARGET=sqlite`
- `E2E_BASE_URL_CONTAINER` - containerized agent-hub webapp url; this is the default full-mode E2E target
- `E2E_LIGHT_BASE_URL` - agent-hub light webapp url to use for test
- `E2E_AGENT_CHAT_CALCULATOR_HITL_AGENT_NAME` - existing agent the test selects as Agent Chat's main LLM before verifying its calculator approval flow (defaults to `calculator agent with hitl`)
- `E2E_AGENT_CHAT_CALCULATOR_HITL_PROMPT` / `E2E_AGENT_CHAT_CALCULATOR_HITL_EXPECTED_RESULT` - portable prompt and result assertion for the Agent Chat calculator HITL test
- `E2E_LLM_MODEL_NAME` - model selected as the Agent Chat document-generation orchestrator or as the main LLM for Agent Hub
- `E2E_AGENT_CHAT_DOCX_PROMPT` / `E2E_AGENT_CHAT_DISABLED_PPTX_PROMPT` - document prompts used after the calculator HITL turn, with the conversation-level generation toggle enabled and disabled
- `E2E_AGENTHUB_TARGET` - optional full-mode target selector, `container_postgres` by default and `sqlite` to force `E2E_BASE_URL`; set it to `sqlite` when no containerized target is configured
- `E2E_ADMIN_POOL_USER1_USERNAME` ... `E2E_ADMIN_POOL_USER5_USERNAME` / `E2E_CONSUMER_POOL_USER1_USERNAME` ... `E2E_CONSUMER_POOL_USER5_USERNAME` - usernames and passwords for the five parallel pool users used by pool-lane E2E runs (`role:FULL_DESIGNER`)
- `E2E_SHARED_USER_USERNAME` - username for the reserved shared DSS user searchable from the My Agent share dialog (`role:FULL_DESIGNER`)
- `E2E_SHARED_USER_PASSWORD` - password for the reserved shared DSS user used by cross-user My Agent E2E coverage



## Evolution

This template will evolve over time.
