ci: deploy examples with SAM stack#517
Conversation
zhongkechen
force-pushed
the
codex/sam-deploy-examples
branch
2 times, most recently
from
00edfd0 to
65c6d03
Compare
zhongkechen
force-pushed
the
codex/sam-deploy-examples
branch
11 times, most recently
from
9dfbb75 to
bfab814
Compare
zhongkechen
force-pushed
the
codex/sam-deploy-examples
branch
from
bfab814 to
92f8b42
Compare
zhongkechen
force-pushed
the
codex/sam-deploy-examples
branch
from
92f8b42 to
3ae645a
Compare
yaythomas
left a comment
yaythomas
left a comment
There was a problem hiding this comment.
This is a massive improvement on the ad-hoc per-function CLI deploy path to a SAM stack, and it deletes a lot of code. Thank you!
The refactor of generate_sam_template.py is really cool too!
which is to say review comments are largely nits and I'm pretty much happy to approve :-)
the broader conceptual question I have is whether we want per-PR, or per-runtime-version stacks. If we want per-runtime, which is what is currently implement, shared per-version stack then only the most recent run is left behind for debugging. (so 1 PR could overwrite another PRs). but if we go per-PR, then we have to think about cleaning up stacks.
I'm not all that opinionated about this either, just flagging this for discussion and happy to go with your preference :-)
| name: Cloud tests | ||
|
|
||
| on: | ||
| pull_request: |
There was a problem hiding this comment.
no paths:...
The old deploy-examples.yml had a paths filter so it only ran when
- packages/aws-durable-execution-sdk-python/src/**
- packages/aws-durable-execution-sdk-python-examples/**,
or the workflow itself changed.
And the old integration-tests.yml was Python 3.13 only.
This new workflow runs on every PR to main regardless of path, on all four Python versions. Not the end of the world, but it could result in unnecessary deployments/actions running.
| STACK_NAME="${PYTHON_PREFIX}-python-examples" | ||
| FUNCTION_NAME_PREFIX="${PYTHON_PREFIX}-" | ||
|
|
||
| STACK_NAME=$(echo "$STACK_NAME" | tr -cd '[:alnum:]-' | cut -c1-128) |
There was a problem hiding this comment.
some of dynamic scripting here we could possibly do in env via format?
The stack name is only keyed on Python version, so all PRs share the same four stacks.
group + cancel-in-progress: false will make it queue rather than clobber existing for simultaneous PRs.
but this means the next PR for that Python version will overwrite everything.
If we want per-PR isolation, then "PR--Python or something"), the small change is to compose the names in env: instead of the run: shell step,
using the pull_request context.
jobs:
example-tests:
env:
# PR runs get PrN; workflow_dispatch and pushes get RunN.
PR_TAG: ${{ github.event.pull_request.number && format('Pr{0}', github.event.pull_request.number) || format('Run{0}', github.run_id) }}
steps:
...
- name: Deploy all examples with SAM
env:
STACK_NAME: ${{ matrix.python-prefix }}-${{ env.PR_TAG }}-python-examples
FUNCTION_NAME_PREFIX: ${{ matrix.python-prefix }}-${{ env.PR_TAG }}-
...
but this also has the negative that per-PR stacks accumulate until they get cleaned up.
We could have a a companion on: pull_request types: [closed] workflow that runs aws cloudformation delete-stack across the four python-prefix values?
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: cloud-tests-${{ matrix.python-prefix }} | ||
| cancel-in-progress: false |
There was a problem hiding this comment.
how about per-PR rather than per-runtime?
concurrency:
group: cloud-tests-${{ matrix.python-prefix }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
this also cancels predecessors for the same PR.
| }, | ||
| } | ||
| }, | ||
| "FunctionNamePrefix": { |
There was a problem hiding this comment.
nit:
maybe make this one required rather than empty, or set a distinctive default like "PythonExamples-"??
if someone (definitely future me!) runs sam deploy manually without setting FunctionNamePrefix, I'd get functions named HelloWorld, Timer, Step, etc.
short, generic, easy to collide with other things in an accountThe CI workflow always sets the prefix, so
CI is safe; the risk is a human running the CONTRIBUTING.md recipe.
| ) | ||
|
|
||
| logger.info("Copying examples from %s", src_dir) | ||
| for file_path in src_dir.rglob("*"): |
There was a problem hiding this comment.
The flattening step copies every file under src/ to build/ and the comment already explains why. But if basenames collide, then ordering is rglob("*") order. When two files collide, whichever comes later wins. Not deterministic across filesystems.
e.g src/foo/step.py alongside src/step/step.py.
could raise on collision?
if not file_path.is_file():
continue
dst = build_dir / file_path.name
if dst.exists() and not dst.samefile(file_path):
raise RuntimeError(
f"Duplicate basename would flatten to {dst}: {file_path}"
)
shutil.copy2(file_path, dst)
|
|
||
| - name: Run Integration Tests | ||
| env: | ||
| AWS_REGION: ${{ env.AWS_REGION }} |
There was a problem hiding this comment.
nit: already set at top-level
| # Examples deployment (from repo root) | ||
| hatch run examples:build | ||
| hatch run examples:deploy "Hello World" | ||
| hatch run examples:generate-sam-template |
There was a problem hiding this comment.
thank you so much for this! awesome to have the CONTRIBUTING guide and the test/README get an update alongside the change :-)
2 participants
Summary
hatch run dev-examples:test, while cloud tests use the deployed stack and an automatically generated function-name map.3.11,3.12,3.13, and3.14jobs actually exercise cloud resources on that runtime.Workflow changes
.github/workflows/cloud-tests.ymlwith workflow display nameCloud tests.mainand viaworkflow_dispatch.example-tests.3.11,3.12,3.13, and3.14.Py311,Py312,Py313, andPy314.${PYTHON_PREFIX}-python-examplesand Lambda function names as${PYTHON_PREFIX}-.cloud-tests-${{ matrix.python-prefix }}, withcancel-in-progress: false, so workflows for the same Python version do not overlap.secrets.TEST_ROLE_ARN.secrets.TEST_ACCOUNT_ID.sam buildandsam deploy.PythonRuntime=python${{ matrix.python-version }}to SAM so each matrix leg deploys functions on the same Python version used by the test runner.secrets.TEST_LAMBDA_EXECUTION_ROLE_ARNas the pre-existing Lambda execution role for every generated function.lambda_endpointinput or theTEST_LAMBDA_ENDPOINTrepository/org variable.LambdaEndpoint) and pytest (LAMBDA_ENDPOINT) so deployed functions and test clients target the same endpoint.LAMBDA_ENDPOINTto later workflow steps when an override is configured, so normal PR runs do not pass an empty endpoint to boto3.PYTEST_FUNCTION_NAME_MAPafter deployment and pass it to the test runner.pytest-xdistusing-n 8 --dist loadfile.Removed workflows and legacy deployment path
.github/workflows/deploy-examples.yml..github/workflows/integration-tests.yml..github/workflows/update-sam-template.yml.packages/aws-durable-execution-sdk-python-examples/template.yaml.packages/aws-durable-execution-sdk-python-examples/cli.py.cli,bootstrap,deploy,invoke,get,history,policy, andlist.SAM/template generation
generate_sam_template.pyto buildtemplate.generated.jsonfromexamples-catalog.json.--outputoption to write the generated template to a custom path.FunctionNamePrefixandLambdaExecutionRoleArntemplate parameters.PythonRuntimetemplate parameter with defaultpython3.13and allowed values forpython3.11,python3.12,python3.13, andpython3.14.Globals.Function.Runtimefrom thePythonRuntimeparameter instead of hard-codingpython3.13.FunctionNamefromFunctionNamePrefixplus the logical ID.Rolefrom theLambdaExecutionRoleArnparameter.DurableConfig,LoggingConfig,Layers,Tracing, and environment settings from the catalog.LambdaEndpointas a SAM parameter with the standard Lambda endpoint default and wire it toAWS_ENDPOINT_URL_LAMBDAfor generated functions.Build/test tooling
scripts/build_examples.pyto build a shared example bundle for SAM packaging.build/directory, installs runtime packages into it, and copies example source files into the Lambda bundle.scripts/generate_function_name_map.pyto create the JSON mapping from example names to qualified$LATESTfunction names.hatch run examples:buildto usebuild_examples.py.hatch run examples:generate-sam-templateto generatetemplate.generated.json.hatch run examples:cleanto remove the examples build directory, generated template, and root.aws-sambuild output.pytest-xdistto the shared test Hatch environment for concurrent cloud test execution.Cloud test runner changes
PYTEST_FUNCTION_NAME_MAP.PYTEST_FUNCTION_NAME_MAP.LAMBDA_ENDPOINTin the cloud runner.LAMBDA_ENDPOINTvalue as unset to avoid boto3Invalid endpointerrors.Documentation and ignore updates
CONTRIBUTING.mdto document the SAM build/deploy flow instead of the old example CLI deploy flow.PythonRuntimeSAM parameter in manual deploy examples.PYTEST_FUNCTION_NAME_MAPas the required cloud-mode test mapping.cloud-tests.yml..aws-sam/output and generatedtemplate.generated.jsonfiles.Testing
hatch fmt --checkpython3 -m py_compile packages/aws-durable-execution-sdk-python-examples/scripts/generate_sam_template.py packages/aws-durable-execution-sdk-python-examples/scripts/generate_function_name_map.py packages/aws-durable-execution-sdk-python-examples/scripts/build_examples.py packages/aws-durable-execution-sdk-python-examples/test/conftest.pyhatch run examples:generate-sam-templateLambdaExecutionRoleArnand noDurableFunctionRoleresource is present.Globals.Function.Runtimeresolves fromPythonRuntime, whose default remainspython3.13.python3 packages/aws-durable-execution-sdk-python-examples/scripts/generate_function_name_map.py --function-name-prefix PythonPr517-sam validate --template-file packages/aws-durable-execution-sdk-python-examples/template.generated.json --region us-west-2sam build --template-file packages/aws-durable-execution-sdk-python-examples/template.generated.jsonhatch run test:examples-integration --help.github/workflows/cloud-tests.ymlgit diff --checkPreviously also ran:
hatch run dev-examples:test -k test_hello_worldhatch run examples:buildDid not run an actual AWS deploy locally.