app log entries do not have request-Id metadata to associate with a request log entry

[timc13]

wild guess, but could it be because the actual request header is X-Appengine-Request-Log-Id and the code looks like it's looking for X-Appengine-Request-Id?

[bryanmau1]

Are you referring to the logs one can see on the developers console? A code snippet or screenshot would be helpful.

[timc13]

I'm referring to the logs i see in Google Cloud Logs. I converted an app from the standard runtime to flexible vm.

Before the conversion, my app logs would show up grouped with the request log. After the conversion they are in a separate app log, which is fine, but there is no request-id in the metadata that I can use to correlate to any request log entries.

[andrewsg]

In the Cloud Logging UI, can you click on the vertical ". . ." on the right-hand side and see if there's a "see logs from same request" or similar functionality exposed? This functionality is based on the "cloud trace ID" (in the metadata of the log) and will be the focus of our log correlation efforts going forward.

[timc13]

@andrewsg - I tried a bunch of things - and still no solution.

If I click "Show entries from same request" on the request log - this gets put in the filter: label:appengine.googleapis.com/request_id:570dc25000ff0265238e01b5860001737e656d69737361727 92d73746167696e670001323031363034313274323334313332000100

If I click "Show entries from same request" on the app log - this gets put in the filter:
label:appengine.googleapis.com/thread_id:140649891129088

I see a protoPayload.traceId in the request log, but traceId doesn't exist in any metadata in the app log. The app log doesn't even have protoPayload.

Hope this helps...

[timc13]

for context, this is our app constructor

def __init__(self, *args, **kwargs):
    super(MyApplication, self).__init__(*args, **kwargs)
    self.router.set_dispatcher(self.__class__.custom_dispatcher)

here's a screenshot of the app log

[timc13]

hey just following up, are you guys able to confirm this is indeed a bug? Or is there something wrong with the way I'm doing things.

[andrewsg]

Not sure quite yet.

[theacodes]

To clarify here:

• App Engine Standard will group logs by request in the Cloud Logging UI. App Engine Standard also only shows the app/request log.
• App Engine Flex, Compute Engine, and Container Engine will not group the logs by request, instead, it services a request log, a stderr log, and a stdout log.

The primary reason for this difference that because there's so much variance in how things can be set up, so there currently isn't a simple way to do log to request correlation. We are investigating this, but we have not yet committed to feature parity.

There are a few workarounds. You can always log in JSON format and log the request ID and use bigquery to analyze by request.

I am closing this issue for the time being.