Skip to content

"ValueError: Unable to parse claims from response" on get() #672

Closed

Description

@cjusko

Hello!

Trying to make any .get() call from my GraphServiceClient results in the following error:

ValueError                                
Traceback (most recent call last)

      1 # GET /users?$top=500
      2 #users = await client.users.by_user_id('cjusko').get()
----> 3 users = await client.users.get()
      4 print(users.value)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/that/lib/python3.11/site-packages/msgraph/generated/users/users_request_builder.py:69, in UsersRequestBuilder.get(self, request_configuration)
     66     raise Exception("Http core is null") 
     67 from ..models.user_collection_response import UserCollectionResponse
---> 69 return await self.request_adapter.send_async(request_info, UserCollectionResponse, error_mapping)

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/that/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:178, in HttpxRequestAdapter.send_async(self, request_info, parsable_factory, error_map)
    175     parent_span.record_exception(REQUEST_IS_NULL)
    176     raise REQUEST_IS_NULL
--> 178 response = await self.get_http_response_message(request_info, parent_span)
    180 response_handler = self.get_response_handler(request_info)
    181 if response_handler:

File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/that/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:543, in HttpxRequestAdapter.get_http_response_message(self, request_info, parent_span, claims)
    541     parent_span.set_attribute("http.response_content_type", content_type)
    542 _get_http_resp_span.end()
--> 543 return await self.retry_cae_response_if_required(resp, request_info, claims)

File ~/.local/share/hatch/env/virtual/that/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:560, in HttpxRequestAdapter.retry_cae_response_if_required(self, resp, request_info, claims)
    558 claims_match = re.search('claims="(.+)"', auth_header_value)
    559 if not claims_match:
--> 560     raise ValueError("Unable to parse claims from response")
    561 response_claims = claims_match.group().split('="')[1]
    562 parent_span.add_event(AUTHENTICATE_CHALLENGED_EVENT_KEY)

ValueError: Unable to parse claims from response

Some context:
This is in virtalenv running
python==3.11.6
and
msgraph-sdk==1.2.0 (but I received the same error on version 1.3.0 as well, tried downgrading to see if it resolved, it did not)

the code I'm running is fairly simple:

from azure.identity import AzureAuthorityHosts
from azure.identity.aio import ClientSecretCredential
from msgraph import GraphServiceClient

auth = {
 # DICT holding 'tenant ID',  'client ID', 'client secret key'
}

credential = ClientSecretCredential(
    auth['tenant'],
    auth['clientId'],
    auth['clientSecret'],
    authority=AzureAuthorityHosts.AZURE_GOVERNMENT
)

scopes = ['https://graph.microsoft.us/.default']
client = GraphServiceClient(credentials=credential, scopes=scopes)

users = await client.users.get() # <--- Error on this line

I am running this in a jupyter notebook, but running in IPython leads to same result. Same result when running in a script and calling like this as well:

async def get_user():
    users = await client.users.get() # <--- Error on this line
    # ...

asyncio.run(get_user())

I'm guessing it's likely an issue with my client setup since I can't seem to find anyone else having this issue. But not sure how else to set it up, I can confirm that my tenant and client information is correct. Maybe it's an issue in how I have it set up to hit AZURE_GOVERNMENT?

Thanks in advance!

Activity

  1. shemogumbe commented on Apr 22, 2024

    @shemogumbe
    Contributor

    Hello, to be able to understand the issue,
    I see you are trying to get a list of users, there is a possibility that:

    • the way the client is instantianted, and the application permissions on your tenant disallow this request.
    • The auth lacks a www-authenticate or token, which could still point to how the client is initialised.

    My suggestions

    • Do you mind trying this on the public cloud.
    • Try counter check with your tenant for the correct tenant_id, client_id and client_secret for use in the credential creation.

    Note
    Kindly counter check the equivalent scope for get users in the government cloud, trying to reproduce your issue got me the error unable to process claims response when using the scope ``

    but worked fine with default authority, and default scope in public cloud:

    No error here:

    redential = ClientSecretCredential(
        auth['tenant_id'],
        client_id=auth['clientId'],
        client_secret=auth['clientSecret'],
        authority=os.getenv('authority'),
       )
    )
    scopes = ['https://graph.microsoft.com/.default']
    client = GraphServiceClient(credentials=credential, scopes=scopes)
    print(client.request_adapter)
    
    
    async def get_users():
        try:
            users = await client.users.get()  # <--- Error on this line
            for user in users.value:
                print(f"User: {user.display_name} {user.id}")
        except APIError as e:
            print(f"Error: {e}")

    Having only replaced scopes = ['https://graph.microsoft.us/.default'] with scopes = ['https://graph.microsoft.com/.default']
    and authority=AzureAuthorityHosts.AZURE_GOVERNMENT with authority='https://login.microsoftonline.com/common'

  2. added
    priority:p2Medium. For a p2 bug, generally have a work-around. Bug SLA <=30 days
    and removed
    priority:p0Blocking issue. Loss of critical functions eg security/privacy violation. Bug SLA<=48hrs
    on Apr 22, 2024
  3. cjusko commented on Apr 24, 2024

    @cjusko
    Author

    making these changes gives back a different error:
    ClientSecretCredential.get_token failed: argument of type 'NoneType' is not iterable
    With this code:

    credential = ClientSecretCredential(
        tenant_id=auth['tenant'],
        client_id=auth['clientId'],
        client_secret=auth['clientSecret'],
        authority='https://login.microsoftonline.com/common' 
    )
    
    scopes = ['https://graph.microsoft.com/.default']
    client = GraphServiceClient(credentials=credential, scopes=scopes)
    
    async def get_me():
        try:
            me = await client.me.get()  # <--- Error on this line
            print(me.value)
        except APIError as e:
            print(f"Error: {e}")
    

    Here's the full traceback:

    ClientSecretCredential.get_token failed: argument of type 'NoneType' is not iterable
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    Cell In[26], line 18
         15 client = GraphServiceClient(credentials=credential, scopes=scopes)
         17 
    ---> 18 me = await client.me.get()
         19 print(me.value)
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/msgraph/generated/users/item/user_item_request_builder.py:156, in UserItemRequestBuilder.get(self, request_configuration)
        153     raise Exception("Http core is null") 
        154 from ...models.user import User
    --> 156 return await self.request_adapter.send_async(request_info, User, error_mapping)
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:178, in HttpxRequestAdapter.send_async(self, request_info, parsable_factory, error_map)
        175     parent_span.record_exception(REQUEST_IS_NULL)
        176     raise REQUEST_IS_NULL
    --> 178 response = await self.get_http_response_message(request_info, parent_span)
        180 response_handler = self.get_response_handler(request_info)
        181 if response_handler:
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:523, in HttpxRequestAdapter.get_http_response_message(self, request_info, parent_span, claims)
        520 if claims:
        521     additional_authentication_context[self.CLAIMS_KEY] = claims
    --> 523 await self._authentication_provider.authenticate_request(
        524     request_info, additional_authentication_context
        525 )
        527 request = self.get_request_from_request_information(
        528     request_info, _get_http_resp_span, parent_span
        529 )
        530 resp = await self._http_client.send(request)
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_abstractions/authentication/base_bearer_token_authentication_provider.py:50, in BaseBearerTokenAuthenticationProvider.authenticate_request(self, request, additional_authentication_context)
         47     request.headers = HeadersCollection()
         49 if not request.headers.contains(self.AUTHORIZATION_HEADER):
    ---> 50     token = await self.access_token_provider.get_authorization_token(
         51         request.url, additional_authentication_context
         52     )
         53     if token:
         54         request.headers.add(f'{self.AUTHORIZATION_HEADER}', f'Bearer {token}')
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/kiota_authentication_azure/azure_identity_access_token_provider.py:106, in AzureIdentityAccessTokenProvider.get_authorization_token(self, uri, additional_authentication_context)
        103     result = self._credentials.get_token(*self._scopes, claims=decoded_claim)
        105 if inspect.isawaitable(result):
    --> 106     result = await result
        107     await self._credentials.close()  # type: ignore
        109 if result and isinstance(result, AccessToken):
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_internal/get_token_mixin.py:93, in GetTokenMixin.get_token(self, claims, tenant_id, enable_cae, *scopes, **kwargs)
         91 if not token:
         92     self._last_request_time = int(time.time())
    ---> 93     token = await self._request_token(
         94         *scopes, claims=claims, tenant_id=tenant_id, enable_cae=enable_cae, **kwargs
         95     )
         96 elif self._should_refresh(token):
         97     try:
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_credentials/client_secret.py:67, in ClientSecretCredential._request_token(self, *scopes, **kwargs)
         66 async def _request_token(self, *scopes: str, **kwargs: Any) -> AccessToken:
    ---> 67     return await self._client.obtain_token_by_client_secret(scopes, self._secret, **kwargs)
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_internal/aad_client.py:48, in AadClient.obtain_token_by_client_secret(self, scopes, secret, **kwargs)
         46 async def obtain_token_by_client_secret(self, scopes: Iterable[str], secret: str, **kwargs) -> AccessToken:
         47     request = self._get_client_secret_request(scopes, secret, **kwargs)
    ---> 48     return await self._run_pipeline(request, **kwargs)
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/aio/_internal/aad_client.py:85, in AadClient._run_pipeline(self, request, **kwargs)
         83 now = int(time.time())
         84 response = await self._pipeline.run(request, retry_on_methods=self._POST, **kwargs)
    ---> 85 return self._process_response(response, now, enable_cae=enable_cae, **kwargs)
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/_internal/aad_client_base.py:161, in AadClientBase._process_response(self, response, request_time, **kwargs)
        158             cache.update_rt(cache_entries[0], content["refresh_token"])
        159             del content["refresh_token"]  # prevent caching a redundant entry
    --> 161 _raise_for_error(response, content)
        163 if "expires_on" in content:
        164     expires_on = int(content["expires_on"])
    
    File ~/.local/share/hatch/env/virtual/hat/hYHKRVU-/hat/lib/python3.11/site-packages/azure/identity/_internal/aad_client_base.py:390, in _raise_for_error(response, content)
        389 def _raise_for_error(response: PipelineResponse, content: Dict) -> None:
    --> 390     if "error" not in content:
        391         return
        393     _scrub_secrets(content)
    
    TypeError: argument of type 'NoneType' is not iterable
    
  4. deanevs commented on May 1, 2024

    @deanevs

    I have the exact same issue, the adapter always get's set to the host: graph.microsoft.com even though the authority is set up with a token for US_GOV??

    Setup ->

            self.credential = ClientSecretCredential(
                client_id=config.client_id,
                tenant_id=config.tenant_id,
                client_secret=config.client_secret,
                authority=AzureAuthorityHosts.AZURE_GOVERNMENT
            )
    
            self.scopes = 'https://graph.microsoft.us/.default'
            allowed_hosts = [NationalClouds.US_GOV]
    
            auth_provider = AzureIdentityAuthenticationProvider(self.credential,
                                                                scopes=[self.scopes],
                                                                allowed_hosts=allowed_hosts)
    
            base_middleware = GraphClientFactory.get_default_middleware(options=None)
            base_middleware.append(DebugHandler())
    
            http_client = GraphClientFactory.create_with_custom_middleware(
                middleware=base_middleware,
                host=NationalClouds.US_GOV,
            )
    
            adapter = GraphRequestAdapter(auth_provider=auth_provider, client=http_client)
            self.client = GraphServiceClient(request_adapter=adapter)
    

    Debug ->

    GET https://graph.microsoft.com/v1.0/users?$orderby=displayName&$select=id,displayName,mail,usageLocation,givenName,surname
    host: graph.microsoft.com
    accept-encoding: gzip, deflate
    connection: keep-alive
    consistencylevel: eventual
    accept: application/json
    authorization: Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6Ijc4ZRWRkZXph....
    user-agent: python-httpx/0.27.0 kiota-python/1.3.1
    ...
    ...
    Response: 401 Unauthorized
    Response headers:
    content-type: application/json
    content-encoding: gzip
    vary: Accept-Encoding
    strict-transport-security: max-age=31536000
    request-id: 4747f7ff-7ad3-4dbc-b2cf-9e3fd6a08c30
    client-request-id: 4747f7ff-7ad3-4dbc-b2cf-9e3fd6a08c30
    x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"UK South","Slice":"E","Ring":"5","ScaleUnit":"001","RoleInstance":"LN2PEPF00010E3B"}}
    www-authenticate: Bearer realm="", authorization_uri="https://login.microsoftonline.com/common/oauth2/authorize", client_id="00000003-0000-0000-c000-000000000000"
    date: Wed, 01 May 2024 09:33:06 GMT
    

    Error ->

      File "\kiota_http\httpx_request_adapter.py", line 543, in get_http_response_message
        return await self.retry_cae_response_if_required(resp, request_info, claims)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "\kiota_http\httpx_request_adapter.py", line 560, in retry_cae_response_if_required
        raise ValueError("Unable to parse claims from response")
    ValueError: Unable to parse claims from response
    
  5. deanevs commented on May 2, 2024

    @deanevs

    Worked it out, you need to set the adapter base_url before passing it to the GraphServiceClient

            adapter = GraphRequestAdapter(auth_provider=auth_provider, client=http_client)
            adapter.base_url = 'https://graph.microsoft.us/v1.0'
            self.client = GraphServiceClient(request_adapter=adapter, scopes=self.scopes)
  6. shemogumbe commented on May 21, 2024

    @shemogumbe
    Contributor

    Moving these two discussions as it may help someone facing a similar issue

  7. isiahzzzz commented on Jun 18, 2024

    @isiahzzzz

    I'm running into this same exact issue.

    Python version 3.12.3, msgraphsdk version 1.4.0.

    I have a Graph class that sets up my GraphServiceClient, I'm also using Application Authentication vs Delegated Auth. This class has a test() method for getting a list of users, and a get_app_only_token() method used to see if I'm getting back an access token, which I am.

    The class:

    class Graph:
        settings: Flask
        client_credential: ClientSecretCredential
        app_client: GraphServiceClient
    
        def __init__(self, config: Flask):
            self.settings = config
            client_id = self.settings.config['CLIENT_ID']
            tenant_id = self.settings.config['TENANT_ID']
            client_secret = self.settings.config['CLIENT_SECRET']
            
            self.client_credential = ClientSecretCredential(tenant_id, 
                                                            client_id, 
                                                            client_secret, 
                                                            authority=AzureAuthorityHosts.AZURE_GOVERNMENT)
            self.app_client = GraphServiceClient(self.client_credential,
                                                scopes=['https://graph.microsoft.us/.default'])
            async def test(self):
                 result = await self.app_client.users.get() ## <-------- error here
                 return result
    
    

    Traceback:

    Traceback (most recent call last):
      File "\app\models\graph.py", line 40, in test
        result = await self.app_client.users.get()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "\apivenv\Lib\site-packages\msgraph\generated\users\users_request_builder.py", line 72, in get
        return await self.request_adapter.send_async(request_info, UserCollectionResponse, error_mapping)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "\apivenv\Lib\site-packages\kiota_http\httpx_request_adapter.py", line 178, in send_async
        response = await self.get_http_response_message(request_info, parent_span)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "\apivenv\Lib\site-packages\kiota_http\httpx_request_adapter.py", line 543, in get_http_response_message
        return await self.retry_cae_response_if_required(resp, request_info, claims)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "\apivenv\Lib\site-packages\kiota_http\httpx_request_adapter.py", line 560, in retry_cae_response_if_required
        raise ValueError("Unable to parse claims from response")
    ValueError: Unable to parse claims from response
    

    I've tried what Dean Evans (@deanevs) recommended but no dice, any advice would be super. Thanks in advance.

  8. zhetao1116 commented on Sep 16, 2024

    @zhetao1116

    Hi Shem Ogumbe (@shemogumbe) ! I am facing the same issue although I am invoking the public API and it happens when the token is invalid or expired.

    class RawAccessTokenProvider:
        """
        A simple credential provider that returns a raw access token for use with Azure SDK clients.
        """
    
        def __init__(self, access_token: str, expires_on: int) -> None:
            self._access_token = access_token
            self._expires_on = expires_on
    
        def get_token(self, *scopes, **kwargs) -> AccessToken:
            return AccessToken(self._access_token, self._expires_on)
    
    
    class MicrosoftCalendarApi(CalendarApiBase):
        """
        A client for interacting with the Microsoft Graph Calendar API.
        This class facilitates interactions with the Microsoft Graph API through the "msgraph" library.
        """
    
        def __init__(self, credentials: Dict[str, Union[str, int]]) -> None:
            super().__init__(credentials)
    
            access_token = credentials.get("access_token")
            client_credentials = RawAccessTokenProvider(access_token, expires_on)
            self.client = GraphServiceClient(credentials=client_credentials, scopes=REQUIRED_SCOPES)
    

    I also tried Dean Evans (@deanevs) approach but it doesn't work unfortunatedly. Any idea how could I fix it to get the error response?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Needs: Attention 👋priority:p2Medium. For a p2 bug, generally have a work-around. Bug SLA <=30 days

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions