PyLimeRPC/README.md

190 lines
7.7 KiB
Markdown
Raw Permalink Normal View History

# PyLimeRPC
2026-09-08 13:06:17 +02:00
**PyLimeRPC** (import package: `pylimerpc`) is a reusable Python library for
interacting with **LimeSurvey** via the RemoteControl JSON-RPC API (LS7,
endpoint `index.php?r=admin/remotecontrol`).
2026-09-08 13:08:19 +02:00
- Python standard library only — no runtime dependencies.
- Usable from cron/URL-triggered data tasks, CLI admin tools and web
dashboards (ASGI).
- Runs on Linux.
## Prerequisites
Before using this library or the example scripts, make sure you have:
1. **Python >= 3.10** on a Linux machine.
2. A **LimeSurvey 7** installation with the **RemoteControl (JSON-RPC) API enabled** —
the endpoint `https://<your-limesurvey-host>/index.php?r=admin/remotecontrol`
must be reachable over HTTPS.
3. A **RemoteControl user account** (username + password) with permissions for the
surveys you want to access.
4. A filled-in **`.env`** file (copy `.env.example`): `LIMESURVEY_BASE_URL`,
`LIMESURVEY_USERNAME`, `LIMESURVEY_PASSWORD`; optionally `LIMESURVEY_TIMEOUT_SECONDS`,
`LIMESURVEY_OWNER_ID`, `LIMESURVEY_GROUP_ID`, `LIMESURVEY_VERIFY_SSL`.
5. Network access from your machine to the LimeSurvey host, with a **valid TLS
certificate** (set `LIMESURVEY_VERIFY_SSL=false` only for a local test instance
with a wrong/self-signed certificate — never in production).
6. To run the tests: `pip install -e ".[dev]"` (adds pytest as a dev extra).
## Install (editable)
```bash
pip install -e .
```
## Structure
```
pylimerpc/
__init__.py # package docstring, LsClient, __version__
config.py # .env reading (standard library, no python-dotenv)
rpc.py # JSON-RPC transport: build request, POST, handle errors
session.py # get_session_key / release_session_key
surveys.py # list_surveys, get_survey_properties, get_survey_by_sid, get_survey_info
responses.py # export_responses (CSV/JSON), export_responses_auto, export_statistics, get_summary, count_responses
tokens.py # token management: add/list/delete participants
errors.py # LsConnectionError, LsApiError, LsAuthError
utils.py # shared helpers
tests/
examples/
pyproject.toml
```
## Configuration
Credentials never live in code or git — only in `.env` (see `.env.example`):
```
LIMESURVEY_BASE_URL=https://example.com/index.php?r=admin/remotecontrol
LIMESURVEY_USERNAME=example_user
LIMESURVEY_PASSWORD=example_password
LIMESURVEY_TIMEOUT_SECONDS=30
```
Note: generated exports (`output/`, `survey_*_responses.*`, `survey_*_statistics.*`,
`*.csv`/`*.xls`/`*.xlsx`/`*.spss`) are gitignored — survey data is never synced to git.
## Usage
```python
from pylimerpc import LsClient, LsConfig
cfg = LsConfig.from_dotenv(".env")
with LsClient(cfg) as ls: # opens a session, releases the key on exit
surveys = ls.list_surveys() # all surveys of this user
props = ls.get_survey_properties(surveys[0]["sid"])
info = ls.get_survey_info(surveys[0]["sid"]) # → SurveyInfo(sid, title, active) or None
survey = ls.get_survey_by_sid(surveys[0]["sid"]) # raw dict from list_surveys, or None
```
Low level (without session management): `RpcClient.from_config(cfg).call(method, params)`.
```python
with LsClient(cfg) as ls:
data = ls.export_responses(sid, doc_type="csv", output_path="output/") # → Path
# automatic format fallback (xlsx → xls → csv) when the LS install lacks a writer:
data, used_format = ls.export_responses_auto(sid) # → (bytes | Path, "xls")
stats = ls.export_statistics(sid) # → bytes
summary = ls.get_summary(sid) # raw dict
counts = ls.count_responses(sid) # → ResponseCounts(completed, incomplete, total)
# extra/custom token attributes (attribute_1, ...) pass through unchanged:
ls.add_participants(sid, [{"email": "a@b.c", "firstname": "A", "attribute_1": "blue"}])
participants = ls.list_participants(sid) # dicts include any attribute_* keys
ls.delete_participants(sid, [participants[0]["tid"]])
```
LS7 RemoteControl methods: `export_responses`, `export_statistics`, `get_summary`,
`add_participants`, `list_participants`, `delete_participants` (base64 exports
are decoded automatically). `count_responses` reads the response counts from
the `get_summary` result (key names differ per LS version; missing/invalid
values count as 0).
## Response counts & summary
`get_summary` returns the raw summary dict of a survey (LS7 RPC `get_summary`,
params `[session_key, sid]`); `count_responses` turns it into a frozen
`ResponseCounts` dataclass:
```python
from pylimerpc import LsClient, LsConfig
cfg = LsConfig.from_dotenv(".env")
with LsClient(cfg) as ls:
summary = ls.get_summary(sid) # raw dict, e.g. {"completed_responses": 14, ...}
counts = ls.count_responses(sid) # ResponseCounts(completed=14, incomplete=41, total=55)
print(counts.completed, counts.incomplete, counts.total)
```
Key names in the summary differ per LS version: `completed_responses`
(fallbacks `responses_completed` / `completed`), `incomplete_responses`
(fallback `in_progress_responses`), and `full_responses` for the total when
present (otherwise `total = completed + incomplete`). Missing or invalid
values count as 0.
Related: `ls.get_survey_info(sid)` → `SurveyInfo(sid, title, active)` or `None`
("does not exist / not active / title" in one call), and
`ls.export_responses_auto(sid)` → `(data, used_format)` with automatic format
fallback (xlsx → xls → csv) when the LS installation lacks a writer.
Errors: `LsConnectionError` (server unreachable/HTTP error), `LsApiError`
(JSON-RPC error, with `rpc_code`/`rpc_message`), `LsAuthError` (login refused).
## Diagnose with curl
Seeing an error? Send the same request by hand:
```bash
curl -i -X POST 'https://host/index.php?r=admin/remotecontrol' \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","method":"get_session_key","params":["USER","PASS"],"id":1}'
```
`-i` shows the HTTP status + headers + body. Note: the alias `/remotecontrol`
returns HTTP 400 — always use `index.php?r=admin/remotecontrol`.
### SSL error: `curl: (60) certificate subject name ... does not match`
The server then presents a certificate for a different hostname (e.g. a local
test box). Diagnose:
```bash
getent hosts ls.example.com # which IP?
echo | openssl s_client -connect ls.example.com:443 -servername ls.example.com 2>/dev/null \
| openssl x509 -noout -subject -ext subjectAltName # which cert?
```
Solutions: use the hostname that IS in the certificate (works with
verification), or for a test instance use `curl -k`. In the library: set
`LIMESURVEY_VERIFY_SSL=false` in `.env` (or `LsConfig(..., verify_ssl=False)`) —
verification stays ON by default. Without this option you get an
`LsConnectionError` with the `ssl.SSLError` as `cause`.
## Examples
See `examples/` — three directly runnable scripts (config from `.env`):
```bash
python examples/example_cron_export.py --sid 12345 --out output/cron # cron task
python examples/example_cli.py surveys # CLI
python examples/example_cli.py export --sid 12345 --out export.csv
python examples/example_cli.py tokens-list --sid 12345
# ASGI dashboard: attach examples.example_asgi:app to your own router
# (dev: uvicorn examples.example_asgi:app --port 8000)
```
## Tests
```bash
pip install -e ".[dev]" # installs pytest as a dev extra
python -m pytest # unit tests (integration tests are skipped by default)
python -m pytest -m integration # integration tests against a real instance
```
Integration tests read `LIMESURVEY_BASE_URL` / `LIMESURVEY_USERNAME` /
`LIMESURVEY_PASSWORD` from the environment and skip when they are not set.
## test.py
`test.py` is the original monolithic script this library grew out of; a next
step is to rewrite it on top of the library.