Allow authentication by JWT Bearer token - #7826
Open
melton-jason wants to merge 30 commits into
Open
Conversation
Triggered by 3561173 on branch refs/heads/issue-5163
melton-jason
marked this pull request as ready for review
March 19, 2026 18:28
acwhite211
requested changes
Mar 20, 2026
acwhite211
left a comment
acwhite211
left a comment
Collaborator
There was a problem hiding this comment.
Nice improvement to our API authentication 👍
github-project-automation
Bot
moved this from 📋Back Log
to Dev Attention Needed
in General Tester Board
Triggered by 7d75944 on branch refs/heads/issue-5163
grantfitzsimmons
self-requested a review
April 6, 2026 14:07
acwhite211
approved these changes
Apr 6, 2026
Member
There was a problem hiding this comment.
In the issue this is resolving, there is a specific request:
We should add support for an API key/token (or similar approach) that can be generated within the security & accounts system and reused.
This was intended to communicate the need for an option in the user interface for generating this. Does it add too much to the scope to integrate this?
Seems we can add a button in the UI for a user in a particular collection to generate this one-time token and save it
grantfitzsimmons
left a comment
grantfitzsimmons
left a comment
Member
There was a problem hiding this comment.
Testing instructions
- Send a POST request to
/accounts/token/containing the username for the user you want to login as, the password, and the desired collection- Ensure the access token is returned, and record the access token for use in future requests
❯ curl -sS -X POST http://localhost/accounts/token/ --data-urlencode "username=spadmin" --data-urlencode "password=test#password" --data-urlencode "collectionid=4"
{"access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb4xsZWN0aW9uIjo0LCJqdGkiOiJhYjcyNzY0MC02MzE2LTQzODItOTAzYy0wMjRmMjUyMGMwOMMiLCJpYXQiOjE3NzU2NzkyMDUsImV4cCI6MTc3NTY4MTAwNX0.wzthbaZzbPb5fkbPhVQ8Qc5R9en2_Ks-55FgnjWbtmI", "expires_in": 1800}%I had to adjust the structure since I had special characters in my password.
- Send a "safe" request (one with a GET method) that requires permissions (such as fetching a specific record or a collection of records) and set the
Authorizationheader of the request toBearer <my_token>, replacing<my_token>with your access token
❯ curl -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiI5ZDkzYjBlMC0wMmU1LTQyMTQtYjE4Mi01NzY4M2Q4MjRkNjYiLCJpYXQiOjE3NzU2NzkwMjAsImV4cCI6MTc3NTY4MDgyMH0.Uub2cBRwap0yCzpRzUooENBsqsx0PSBcV7vgRGjg4nQ" "http://localhost/api/specify_rows/institution/?fields=name,divisions__name,divisions__disciplines__name,divisions__disciplines__collections__collectionname"
[["University of Kansas Biodiversity Institute", "Entomology", "Botany", "KUEntoPlant"], ["University of Kansas Biodiversity Institute", "Entomology", "Botany (2)", "Collection"], ["University of Kansas Biodiversity Institute", "Entomology", "Botany (2)", "Collection2"], ["University of Kansas Biodiversity Institute", "Entomology", "Botany (2)", "Collection3"], ["University of Kansas Biodiversity Institute", "Entomology", "Entomology", "KUEntoPinned"], ["University of Kansas Biodiversity Institute", "Entomology", "Herpetology", null], ["University of Kansas Biodiversity Institute", "Entomology", "Invertebrate Paleontology", "KUEntoFossil"]]%
- Ensure the request can be fulfilled and the correct data is returned
- Send an "unsafe" request (one with a POST, PUT, DELETE method, such as creating a new record, updating/delete a record, etc.) and set the
Authorizationheader of the request toBearer <my_token>, replacing<my_token>with your access token
❯ curl -X POST \
-H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiI5ZDkzYjBlMC0wMmU1LTQyMTQtYjE4Mi01NzY4M2Q4MjRkNjYiLCJpYXQiOjE3NzU2NzkwMjAsImV4cCI6MTc3NTY4MDgyMH0.Uub2cBRwap0yCzpRzUooENBsqsx0PSBcV7vgRGjg4nQ" \
-H "Content-Type: application/json" \
-d '{
"agenttype": 1,
"lastname": "Fitzsimmons",
"firstname": "Grant"
}' \
http://localhost/api/specify/agent/
{"id": 10482, "abbreviation": null, "agenttype": 1, "date1": null, "date1precision": null, "date2": null, "date2precision": null, "dateofbirth": null, "dateofbirthprecision": null, "dateofdeath": null, "dateofdeathprecision": null, "datetype": null, "email": null, "firstname": "Grant", "guid": "6c4dde2e-5ce4-4591-b4d5-3c537e6adc68", "initials": null, "integer1": null, "integer2": null, "interests": null, "jobtitle": null, "lastname": "Fitzsimmons", "middleinitial": null, "remarks": null, "suffix": null, "text1": null, "text2": null, "text3": null, "text4": null, "text5": null, "timestampcreated": "2026-04-08T15:18:22.812123", "timestampmodified": "2026-04-08T15:18:22.812132", "title": null, "url": null, "verbatimdate1": null, "verbatimdate2": null, "version": 0, "collcontentcontact": null, "colltechcontact": null, "createdbyagent": "/api/specify/agent/3/", "division": null, "instcontentcontact": null, "insttechcontact": null, "modifiedbyagent": null, "organization": null, "specifyuser": null, "addresses": [], "orgmembers": "/api/specify/agent/?organization=10482", "agentattachments": [], "agentgeographies": [], "identifiers": [], "agentspecialties": [], "variants": [], "collectors": "/api/specify/collector/?agent=10482", "components": "/api/specify/component/?identifiedby=10482", "groups": [], "members": "/api/specify/groupperson/?member=10482", "resource_uri": "/api/specify/agent/10482/"}% ~ ❯
Ensure the request can be fulfilled and the requested operation successfully performed
Generate an access token with a short time to live (lifespan)-- such as 30 seconds, 1 minute, 3 minutes, etc.
curl -X POST \
--data-urlencode "username=spadmin" \
--data-urlencode "password=test#password" \
--data-urlencode "collectionid=4" \
--data-urlencode "expires=10" \
http://localhost/accounts/token/
- Wait for the token to expire and the time to live to elapse
- Send a privileged request using the access token and ensure the request fails and the response has a 401 status code
After 1 minute:
curl -X POST \
-H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiJmMTdlMDkzMy04NDc2LTRkOTUtOTY0Ni1kNTQwYmMyZjY5ODYiLCJpYXQiOjE3NzU2Nzk2MDIsImV4cCI6MTc3NTY3OTYxMn0.v-I48oKuFWbcK7FtyK9sECA_je0dvR1wK-9FQBOlQAU" \
-H "Content-Type: application/json" \
-d '{
"agenttype": 1,
"lastname": "Melton",
"firstname": "Jason"
}' \
http://localhost/api/specify/agent/
Invalid access token%
- Revoke an active access token that is still going to be live by the time the next step is performed using the
/accounts/token/revoke/
❯ curl -X POST \
-H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiI5ZDkzYjBlMC0wMmU1LTQyMTQtYjE4Mi01NzY4M2Q4MjRkNjYiLCJpYXQiOjE3NzU2NzkwMjAsImV4cCI6MTc3NTY4MDgyMH0.Uub2cBRwap0yCzpRzUooENBsqsx0PSBcV7vgRGjg4nQ" \
--data-urlencode "access_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiJiY2UxNTc5MS1mMWIwLTQ3MjUtYjQyOC00YTlkMmI1MzQzYzciLCJpYXQiOjE3NzU2Nzk3MDksImV4cCI6MTc3NTY4MTUwOX0.XXHAGZLYb6QUY4wlDBItasXKZHgr1v5akoPAsJhdRIo" \
http://localhost/accounts/token/revoke/
Send a privileged request using the revoked access token and ensure the request fails and the response has a 401 status code
Attempt to generate an access token to a collection that exists but that the user does not have access to
Ensure server returns with a 403 Forbidden status response and does not generate the access token
This user doesn't have access to log into the KUEntoPinned collection, yet I could get a token and create a record:
❯ curl -sS -X POST http://localhost/accounts/token/ --data-urlencode "username=jthomas" --data-urlencode "password=testuser" --data-urlencode "collectionid=4"
{"access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjIsInVzZXJuYW1lIjoianRob21hcyIsImNvbGxlY3Rpb24iOjQsImp0aSI6IjU3MWI0M2U1LTVlZDQtNDdhMS1iYTlmLWJkOGRhYzE1ZTc1NiIsImlhdCI6MTc3NTY3OTgyOCwiZXhwIjoxNzc1NjgxNjI4fQ.8dgWMzVkAss_J10J1f7P_AONMrEj3_nGaKpmnXBJ4QA", "expires_in": 1800}% ❯ curl -X POST \
-H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiI5ZDkzYjBlMC0wMmU1LTQyMTQtYjE4Mi01NzY4M2Q4MjRkNjYiLCJpYXQiOjE3NzU2NzkwMjAsImV4cCI6MTc3NTY4MDgyMH0.Uub2cBRwap0yCzpRzUooENBsqsx0PSBcV7vgRGjg4nQ" \
--data-urlencode "access_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BlbnRvYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiJiY2UxNTc5MS1mMWIwLTQ3MjUtYjQyOC00YTlkMmI1MzQzYzciLCJpYXQiOjE3NzU2Nzk3MDksImV4cCI6MTc3NTY4MTUwOX0.XXHAGZLYb6QUY4wlDBItasXKZHgr1v5akoPAsJhdRIo" \
http://localhost/accounts/token/revoke/
❯ curl -X POST \
-H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjIsInVzZXJuYW1lIjoianRob21hcyIsImNvbGxlY3Rpb24iOjQsImp0aSI6IjU3MWI0M2U1LTVlZDQtNDdhMS1iYTlmLWJkOGRhYzE1ZTc1NiIsImlhdCI6MTc3NTY3OTgyOCwiZXhwIjoxNzc1NjgxNjI4fQ.8dgWMzVkAss_J10J1f7P_AONMrEj3_nGaKpmnXBJ4QA" \
-H "Content-Type: application/json" \
-d '{
"agenttype": 1,
"lastname": "Melton",
"firstname": "Jason"
}' \
http://localhost/api/specify/agent/
{"id": 10483, "abbreviation": null, "agenttype": 1, "date1": null, "date1precision": null, "date2": null, "date2precision": null, "dateofbirth": null, "dateofbirthprecision": null, "dateofdeath": null, "dateofdeathprecision": null, "datetype": null, "email": null, "firstname": "Jason", "guid": "3a8a73a8-b2bf-41cd-bb6b-49f88e86c676", "initials": null, "integer1": null, "integer2": null, "interests": null, "jobtitle": null, "lastname": "Melton", "middleinitial": null, "remarks": null, "suffix": null, "text1": null, "text2": null, "text3": null, "text4": null, "text5": null, "timestampcreated": "2026-04-08T15:24:19.966392", "timestampmodified": "2026-04-08T15:24:19.966593", "title": null, "url": null, "verbatimdate1": null, "verbatimdate2": null, "version": 0, "collcontentcontact": null, "colltechcontact": null, "createdbyagent": "/api/specify/agent/6049/", "division": null, "instcontentcontact": null, "insttechcontact": null, "modifiedbyagent": null, "organization": null, "specifyuser": null, "addresses": [], "orgmembers": "/api/specify/agent/?organization=10483", "agentattachments": [], "agentgeographies": [], "identifiers": [], "agentspecialties": [], "variants": [], "collectors": "/api/specify/collector/?agent=10483", "components": "/api/specify/component/?identifiedby=10483", "groups": [], "members": "/api/specify/groupperson/?member=10483", "resource_uri": "/api/specify/agent/10483/"}% ~ ❯ 
Contributor
Author
@grantfitzsimmons I actually wasn't able to directly recreate the Issue you've described: I always correctly receive a 403 Forbidden response when both generating a token for a collection a user does not have access to, and when using a token that's scoped to a collection the user doesn't have access to (such as when collection access is revoked while a token is still live, or when the token is forged because the SECRET_KEY was leaked). However, I do think I understand what happened to cause this behavior. Take for example the following scenario:
The below video demonstrates this behavior in the application: Screen.Recording.2026-04-22.at.8.39.33.AM.movSQL Queries to retreive permissionsQuery to fetch all permissions assigned by roles: SELECT user.SpecifyUserID AS 'User ID',
user.Name AS 'User Name',
sprole.Name AS 'Role Name',
rolepol.resource AS 'Role Resource',
rolepol.action AS 'Role Action',
rolcol.UserGroupScopeID AS 'Role Collection ID',
rolcol.CollectionName AS 'Role Collection Name'
FROM specifyuser user
LEFT OUTER JOIN spuserrole ON spuserrole.specifyuser_id = user.SpecifyUserID
LEFT OUTER JOIN sprole ON sprole.id = spuserrole.role_id
LEFT OUTER JOIN sprolepolicy rolepol ON rolepol.role_id = sprole.id
LEFT OUTER JOIN collection rolcol ON rolcol.UserGroupScopeID = sprole.collection_id;Query to fetch all permissions assigned as the user level: SELECT user.SpecifyUserID AS 'User ID',
policy.resource AS 'User Policy Resource',
policy.action AS 'User Policy Action',
col.UserGroupScopeID AS 'Collection ID',
col.collectionname AS 'User Policy Collection Name'
FROM specifyuser user
JOIN spuserpolicy policy ON policy.specifyuser_id = user.SpecifyUserID
LEFT OUTER JOIN collection col ON col.UserGroupScopeID = policy.collection_id;In other words, the "Enable Collection Access" checkbox only removes the User-Level policy that grants collection access. I assume the intended behavior is for it to also remove all assigned roles within that collection? |
|
Warning One or more dependencies are approaching or past End-of-Life. |
Contributor
📝 WalkthroughWalkthroughChangesJWT authentication now supports collection-scoped token issuance, Bearer-token validation, Redis-backed revocation, and middleware-based request authentication. Collection context resolution preserves authenticated request values. Runtime secret configuration now supports generated fallback keys. JWT Authentication
🚥 Pre-merge checks | ✅ 4 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| try: | ||
| collection = Collection.objects.get(id=collection_id) | ||
| except Collection.DoesNotExist: | ||
| return http.HttpResponseBadRequest(f'collection {collection_id} does not exist') |
| from specifyweb.backend.accounts.permissions_types import InviteLinkPT, SetPasswordPT, SetUserAgentsPT, Sp6AdminPT, UserOICProvidersPT | ||
| from specifyweb.backend.accounts.types import ExternalUser, InviteToken, OAuthLogin, ProviderConf, ProviderInfo | ||
| from specifyweb.middleware.general import require_GET, require_http_methods | ||
| from specifyweb.backend.context.views import has_collection_access, set_collection_cookie, users_collections_for_sp7 |
| @overload | ||
| def _get_string(key: str, delete_key: bool, decode_responses: True) -> str | None: ... | ||
| def _get_string(key: str, delete_key: bool, | ||
| decode_responses: True) -> str | None: ... |
| @overload | ||
| def _get_string(key: str, delete_key: bool, decode_responses: False) -> bytes | None: ... | ||
| def _get_string(key: str, delete_key: bool, | ||
| decode_responses: False) -> bytes | None: ... |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Dockerfile`:
- Around line 233-243: The Dockerfile-generated settings/secret_key.py must not
define or bake a fallback JWT key via DEFAULT_KEY. Remove build-time key
generation and require SECRET_KEY from the runtime secret store, while
preserving a single runtime value shared by all application processes; update
the new_key/SECRET_KEY flow accordingly and reject missing or placeholder
values.
In `@specifyweb/backend/accounts/access_token_utils.py`:
- Around line 29-37: Update the JWT payload in the token-generation flow to set
the “sub” claim from user.id as a string. In the authentication middleware,
convert the decoded subject back to an integer before permission checks and
model queries, preserving existing numeric behavior downstream.
In `@specifyweb/backend/accounts/middleware.py`:
- Around line 33-34: Update the WWW-Authenticate header assignment in the
authentication middleware to prefix the existing challenge parameters with the
Bearer authentication scheme. Preserve the current error and error_description
values while ensuring clients receive a valid Bearer challenge.
In `@specifyweb/backend/cache/redis/store.py`:
- Around line 41-42: Update key_exists to format the supplied key through the
same key-formatting mechanism used by set_string before calling _key_exists,
ensuring token_is_revoked checks the Redis key written by revoke_access_token.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: f338793a-98ef-4b16-ad77-977af62d4365
📒 Files selected for processing (10)
Dockerfilespecifyweb/backend/accounts/access_token_utils.pyspecifyweb/backend/accounts/middleware.pyspecifyweb/backend/accounts/urls.pyspecifyweb/backend/accounts/views.pyspecifyweb/backend/cache/redis/store.pyspecifyweb/backend/cache/redis/utils.pyspecifyweb/backend/context/middleware.pyspecifyweb/backend/context/views.pyspecifyweb/settings/__init__.py
Comment on lines
+233
to
+243
| RUN cat <<EOF > settings/secret_key.py | ||
| import os | ||
| DEFAULT_KEY="$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 50)" | ||
| CURRENT_KEY=os.getenv("SECRET_KEY") | ||
|
|
||
| if CURRENT_KEY is None or CURRENT_KEY.strip() == "" or CURRENT_KEY.strip().replace(" ", "_") == "change_this_to_some_unique_random_string": | ||
| new_key = DEFAULT_KEY | ||
| else: | ||
| new_key = CURRENT_KEY | ||
|
|
||
| SECRET_KEY=new_key |
Contributor
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Do not bake the JWT signing key into the image.
Line 235 writes the fallback key into an image layer. Anyone who can inspect or obtain the image can recover the signing key. All deployments from that image also share the key.
Require SECRET_KEY from the runtime secret store. If automatic generation is required, generate and persist it in an access-controlled runtime secret shared by all application processes.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@Dockerfile` around lines 233 - 243, The Dockerfile-generated
settings/secret_key.py must not define or bake a fallback JWT key via
DEFAULT_KEY. Remove build-time key generation and require SECRET_KEY from the
runtime secret store, while preserving a single runtime value shared by all
application processes; update the new_key/SECRET_KEY flow accordingly and reject
missing or placeholder values.
Comment on lines
+29
to
+37
| "sub": user.id, | ||
| "username": user.name, | ||
| "collection": collection_id, | ||
| "jti": jti, | ||
| "iat": datetime.now(timezone.utc), | ||
| "exp": datetime.now(timezone.utc) + timedelta(seconds=expires_in) | ||
| } | ||
| token = jwt.encode(jwt_payload, settings.SECRET_KEY, | ||
| algorithm=AUTH_TOKEN_ALGORITHMS[0]) |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python -m pip install --quiet 'PyJWT==2.12.1'
python - <<'PY'
import time
import jwt
now = int(time.time())
token = jwt.encode(
{"sub": 1, "iat": now, "exp": now + 60, "jti": "test"},
"test-key",
algorithm="HS256",
)
try:
jwt.decode(token, "test-key", algorithms=["HS256"])
except jwt.InvalidTokenError as error:
print(type(error).__name__, error)
PYRepository: specify/specify7
Length of output: 191
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- target file ---'
sed -n '1,110p' specifyweb/backend/accounts/access_token_utils.py
printf '%s\n' '--- middleware references ---'
rg -n -C 4 'jwt\.decode|AUTH_TOKEN|payload|["'"'"']sub["'"'"']' specifyweb/backend/accounts/middleware.py specifyweb/backend/accounts 2>/dev/null || true
printf '%s\n' '--- user model definition ---'
rg -n -C 5 'class Specifyuser|id\s*=|AutoField|BigAutoField' specifyweb -g '*.py' | head -200
printf '%s\n' '--- dependency declarations ---'
rg -n -i -C 2 'pyjwt|jwt' . -g 'requirements*.txt' -g 'pyproject.toml' -g 'setup.py' -g 'setup.cfg' -g 'Pipfile*' -g '*.lock' 2>/dev/null || trueRepository: specify/specify7
Length of output: 29084
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- middleware ---'
cat -n specifyweb/backend/accounts/middleware.py
printf '%s\n' '--- Specifyuser model candidates ---'
rg -n -l 'class Specifyuser' specifyweb -g '*.py' | while read -r file; do
printf '\n--- %s ---\n' "$file"
rg -n -C 12 'class Specifyuser' "$file"
done
printf '%s\n' '--- token generation and middleware tests/usages ---'
rg -n -C 5 'generate_access_token|get_token_from_request|request\.specify_user|check_permission_targets' specifyweb -g '*.py' | head -300Repository: specify/specify7
Length of output: 30094
🌐 Web query:
PyJWT 2.12.1 decode verify_sub subject must be a string source
💡 Result:
In PyJWT 2.10.0 and later versions, including 2.12.1, the validation logic for the sub (subject) claim was updated to enforce that, if the sub claim is present in a JWT payload, it must be a string [1][2]. This change ensures compliance with the JWT specification (RFC 7519), which defines the sub claim as a case-sensitive string [1][3]. When you call jwt.decode(), PyJWT automatically performs this check if verify_sub is enabled (it is enabled by default) [4][5]. If the sub claim is present and is not a string, PyJWT will raise an jwt.exceptions.InvalidSubjectError: Subject must be a string exception [1][4][5]. If you are encountering this error and need to maintain compatibility with tokens that contain non-string sub claims, you can disable this specific validation by updating your decode call to set verify_sub to False in the options [1][3]: jwt.decode(token, key, options={"verify_sub": False}) Alternatively, if you intend to verify the subject against an expected value, ensure your token payload uses a string for the sub field, or use the subject parameter in the decode function to perform a specific match check [1][3].
Citations:
- 1: sub must be a string now? jpadilla/pyjwt#1017
- 2: jpadilla/pyjwt@2.9.0...2.12.1
- 3: sub must be a string now? jpadilla/pyjwt#1017
- 4: https://github.com/jpadilla/pyjwt/blob/master/jwt/api_jwt.py
- 5: https://pyjwt.readthedocs.io/en/2.12.0/api.html
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- collection access and permission call path ---'
rg -n -C 12 'def has_collection_access|def check_permission_targets' specifyweb/backend -g '*.py'
printf '%s\n' '--- structural verifier ---'
python3 - <<'PY'
import ast
from pathlib import Path
access_path = Path("specifyweb/backend/accounts/access_token_utils.py")
middleware_path = Path("specifyweb/backend/accounts/middleware.py")
access = ast.parse(access_path.read_text())
middleware = ast.parse(middleware_path.read_text())
generate = next(
node for node in ast.walk(access)
if isinstance(node, ast.FunctionDef) and node.name == "generate_access_token"
)
sub_assignments = [
node for node in ast.walk(generate)
if isinstance(node, ast.Assign)
and isinstance(node.value, ast.Dict)
and any(
isinstance(key, ast.Constant) and key.value == "sub"
for key in node.value.keys
)
]
sub_value = None
for assignment in sub_assignments:
for key, value in zip(assignment.value.keys, assignment.value.values):
if isinstance(key, ast.Constant) and key.value == "sub":
sub_value = ast.dump(value)
print("generated_sub_expression:", sub_value)
decode_calls = [
node for node in ast.walk(access)
if isinstance(node, ast.Call)
and isinstance(node.func, ast.Attribute)
and node.func.attr == "decode"
]
print("decode_call_count:", len(decode_calls))
middleware_sub_assignments = [
node for node in ast.walk(middleware)
if isinstance(node, ast.Assign)
and any(isinstance(target, ast.Name) and target.id == "user_id"
for target in node.targets)
]
print("middleware_user_id_expression:", [
ast.dump(node.value) for node in middleware_sub_assignments
])
PYRepository: specify/specify7
Length of output: 4248
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- permission query implementation ---'
rg -n -C 20 'def query_pt|def query\(' specifyweb/backend/permissions -g '*.py'
printf '%s\n' '--- model/query consumers of token sub ---'
rg -n -C 4 'token\[[\"'\"']sub[\"'\"']\]|user_id\s*=\s*token' specifyweb -g '*.py'Repository: specify/specify7
Length of output: 7726
Serialize the JWT sub claim as a string.
PyJWT 2.12.1 rejects a non-string sub claim during jwt.decode(). Set "sub": str(user.id). Convert the claim to int in specifyweb/backend/accounts/middleware.py before permission checks and model queries.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@specifyweb/backend/accounts/access_token_utils.py` around lines 29 - 37,
Update the JWT payload in the token-generation flow to set the “sub” claim from
user.id as a string. In the authentication middleware, convert the decoded
subject back to an integer before permission checks and model queries,
preserving existing numeric behavior downstream.
Comment on lines
+33
to
+34
| response = HttpResponse('Invalid access token', status=401) | ||
| response["WWW-Authenticate"] = 'error=\"invalid_token\", error_description=\"The access token is expired, revoked, or invalid\"' |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Prefix the authentication challenge with Bearer.
The current WWW-Authenticate value starts with error. It is not a valid Bearer authentication challenge. Clients cannot reliably identify the authentication scheme.
- response["WWW-Authenticate"] = 'error="invalid_token", error_description="The access token is expired, revoked, or invalid"'
+ response["WWW-Authenticate"] = 'Bearer error="invalid_token", error_description="The access token is expired, revoked, or invalid"'📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| response = HttpResponse('Invalid access token', status=401) | |
| response["WWW-Authenticate"] = 'error=\"invalid_token\", error_description=\"The access token is expired, revoked, or invalid\"' | |
| response = HttpResponse('Invalid access token', status=401) | |
| response["WWW-Authenticate"] = 'Bearer error=\"invalid_token\", error_description=\"The access token is expired, revoked, or invalid\"' |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@specifyweb/backend/accounts/middleware.py` around lines 33 - 34, Update the
WWW-Authenticate header assignment in the authentication middleware to prefix
the existing challenge parameters with the Bearer authentication scheme.
Preserve the current error and error_description values while ensuring clients
receive a valid Bearer challenge.
Comment on lines
+41
to
+42
| def key_exists(key: str) -> bool: | ||
| return _key_exists(key) |
Contributor
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Format revocation keys before checking Redis.
set_string writes revocation entries through the formatted-key store interface. key_exists checks the unformatted key. token_is_revoked therefore does not find the entry written by revoke_access_token, so revoked tokens remain valid.
Proposed fix
def key_exists(key: str) -> bool:
- return _key_exists(key)
+ return _key_exists(format_key(key))📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def key_exists(key: str) -> bool: | |
| return _key_exists(key) | |
| def key_exists(key: str) -> bool: | |
| return _key_exists(format_key(key)) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@specifyweb/backend/cache/redis/store.py` around lines 41 - 42, Update
key_exists to format the supplied key through the same key-formatting mechanism
used by set_string before calling _key_exists, ensuring token_is_revoked checks
the Redis key written by revoke_access_token.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5163
This PR allows a new method of authenticating with the API. Specifically, this PR allows authentication via JWT Bearer tokens.
Previously, the required workflow to authenticate via the API required:
/context/login/).X-CSRFTokenheader with the each unsafe request made to the backend/context/login/with the user's name, password, and collection idFor an example, the prior workflow can be modeled by something like the following Python pseudo code (inspired by the requests library):
With the new approach, users of the API only require:
/context/login/GET endpoint)/accounts/token/with their username, password, and desired collection id to retrieve an access tokenOverview
Acquiring an Access Token
Access tokens can be acquired by sending a POST request to
/accounts/token/and passing theusername,password,collectionid, and optionallyexpires.If the request is successful, the access token is retrievable by the
access_tokenkey in the response's JSON output.By default, access tokens last 1800 seconds (30 minutes), but their lifespan can be configured (see below Setting a token's lifespan).
Example with
curl:In the above case, the resulting access token is
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjEsInVzZXJuYW1lIjoic3BmaXNoYWRtaW4iLCJjb2xsZWN0aW9uIjo0LCJqdGkiOiI2NWMzMmYwNy1hYTMzLTQxN2MtYjI2Ny02MDQwOGQyOTQ0ZjYiLCJpYXQiOjE3NzM5NDUxODIsImV4cCI6MTc3Mzk0Njk4Mn0.s3FTc9EeObiSmm9FLywlpdHkXMKiAob1QuVkW8pp3_o.Example with Python requests
Setting a token's lifespan
By default, access tokens last 1800 seconds (30 minutes).
An access token's lifespan can be set by passing in an
expiresattribute when requesting the token. The backend expectsexpiresto be in seconds.Once an access token expires, it will not be usable and a new access token needs to be generated.
An access token can be made invalid regardless of its expiration time by revoking it (see Revoking an Access Token).
Example of generating an access token that's live for 5 minutes (300 seconds) with curl:
curl -d "username=myuser&password=mypass&collectionid=4&expires=300" http://localhost/accounts/token/Example of generating an access token that's live for 5 minutes (300 seconds) with Python requests:
Using an Access Token
Once an access token is generated, it can be used by passing it in subsequent requests by the
Authorizationheader with theBearerscheme.In other words, the general form of the
Authorizationheader should look likeAuthorizarion: Bearer <my_token>, where<my_token>is replaced with the access token.Example of fetching the institutional hierarchy (Institution, Division, Discipline, Collection) for each Collection using curl:
Example of creating a new agent using Python requests:
If the token is invalid, expired, or revoked then Specify will return a 401 Unauthorized response with the
WWW-Authenticateheaders indicating an invalid token:Revoking an Access Token
An access token can be made invalid by revoking it. To revoke a token, a POST request can be sent to
/accounts/token/revoke/where the request body includes the token to be revoked under anaccess_tokenkey.The client must be authenticated (whether via the previous session authentication or by access token) to make the request.
The same token that is being used to authorize the request to revoke an access token can be revoked. That is, an token can revoke itself.
Below is a snippet of Python that shows how to revoke an access token:
If the token to be revoked is invalid or expired, a 400 Bad Request is returned by the server.
OpenAPI
If you need a reminder/refresher about the token endpoints, they are documented and available to try out at the instance's Operations API page (accessible via User Tools)
Checklist
self-explanatory (or properly documented)
TODO
Testing instructions
In your testing, you can use any client that supports sending HTTP/HTTPS requests: curl, Postman, any supported programming language, etc.
Send a POST request to
/accounts/token/containing the username for the user you want to login as, the password, and the desired collectionEnsure the access token is returned, and record the access token for use in future requests
Send a "safe" request (one with a GET method) that requires permissions (such as fetching a specific record or a collection of records) and set the
Authorizationheader of the request toBearer <my_token>, replacing<my_token>with your access tokenEnsure the request can be fulfilled and the correct data is returned
Send an "unsafe" request (one with a POST, PUT, DELETE method, such as creating a new record, updating/delete a record, etc.) and set the
Authorizationheader of the request toBearer <my_token>, replacing<my_token>with your access tokenEnsure the request can be fulfilled and the requested operation successfully performed
Generate an access token with a short time to live (lifespan)-- such as 30 seconds, 1 minute, 3 minutes, etc.
Wait for the token to expire and the time to live to elapse
Send a privileged request using the access token and ensure the request fails and the response has a 401 status code
Revoke an active access token that is still going to be live by the time the next step is performed using the
/accounts/token/revoke/Send a privileged request using the revoked access token and ensure the request fails and the response has a 401 status code
Attempt to generate an access token to a collection that exists but that the user does not have access to
Ensure server returns with a 403 Forbidden status response and does not generate the access token
Summary by CodeRabbit