Skip to content

update from 1.9.0 to 1.9.1 causes kiota_authentication_azure._exceptions.HTTPError: Valid url scheme and host required #470

Closed

Description

Hi all,

one of my CI jobs checking user status in azure started failing yesterday following upgrade of the kiota python packages.
I've pinned it back down to 1.9.0 so am not stuck but I'd like to know if I am doing something wrong in my script.

Environement:

  • Linux and Mac.
  • Python 3.12.5
    -- microsoft-kiota-abstractions==1.9.1
    -- microsoft-kiota-authentication-azure==1.9.1
    -- microsoft-kiota-http==1.9.1
    -- microsoft-kiota-serialization-form==1.9.1
    -- microsoft-kiota-serialization-json==1.9.1
    -- microsoft-kiota-serialization-multipart==1.9.1
    -- microsoft-kiota-serialization-text==1.9.1
    -- msgraph-core==1.1.2
    -- msgraph-sdk==1.5.4
    -- urllib3==2.2.2
Traceback (most recent call last):
  File "/Users/clecoz/work/cmf_lib/cmflib_main/cmflib/test_kiota.py", line 52, in <module>
    asyncio.run(_get_user_status('clecoz'))
  File "/Users/clecoz/.pyenv/versions/3.12.8/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/clecoz/.pyenv/versions/3.12.8/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/clecoz/.pyenv/versions/3.12.8/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/clecoz/work/cmf_lib/cmflib_main/cmflib/test_kiota.py", line 38, in _get_user_status
    response = await client.users.get(request_configuration=request_configuration)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/clecoz/.pyenv/versions/ldap_3.12.8/lib/python3.12/site-packages/msgraph/generated/users/users_request_builder.py", line 72, in get
    return await self.request_adapter.send_async(request_info, UserCollectionResponse, error_mapping)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/clecoz/.pyenv/versions/ldap_3.12.8/lib/python3.12/site-packages/kiota_http/httpx_request_adapter.py", line 186, in send_async
    response = await self.get_http_response_message(request_info, parent_span)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/clecoz/.pyenv/versions/ldap_3.12.8/lib/python3.12/site-packages/kiota_http/httpx_request_adapter.py", line 595, in get_http_response_message
    await self._authentication_provider.authenticate_request(
  File "/Users/clecoz/.pyenv/versions/ldap_3.12.8/lib/python3.12/site-packages/kiota_abstractions/authentication/base_bearer_token_authentication_provider.py", line 50, in authenticate_request
    token = await self.access_token_provider.get_authorization_token(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/clecoz/.pyenv/versions/ldap_3.12.8/lib/python3.12/site-packages/kiota_authentication_azure/azure_identity_access_token_provider.py", line 78, in get_authorization_token
    raise exc
kiota_authentication_azure._exceptions.HTTPError: Valid url scheme and host required

Quick test script I wrote to investigate: following works with 1.9.0 but give above error with 1.9.1

#!/usr/bin/env python3

import asyncio
from msgraph import GraphServiceClient
from msgraph.generated.users.users_request_builder import UsersRequestBuilder
from msgraph.generated.groups.groups_request_builder import GroupsRequestBuilder
from azure.identity.aio import ClientSecretCredential



DEFAULT_GRAPH = 'https://graph.microsoft.com/.default'
DEFAULT_TENANT = 'tenant_redacted'

username='uname_redacted'
password='password_redacted'


credentials = ClientSecretCredential(DEFAULT_TENANT, username, password)
scopes=[DEFAULT_GRAPH]

client = GraphServiceClient(credentials=credentials, scopes=scopes)


async def _get_user_status(userid:str) -> dict:
    query_params = UsersRequestBuilder.UsersRequestBuilderGetQueryParameters(
        search=[
            f'("displayName:{userid}" OR "mail:{userid}" OR "userPrincipalName:{userid}"\
                    OR "givenName:{userid}" OR "surName:{userid}" OR "otherMails:{userid}")'
        ],
        filter="accountEnabled eq true",
    )
    request_configuration = (
        UsersRequestBuilder.UsersRequestBuilderGetRequestConfiguration(
            query_parameters=query_params,
        )
    )
    request_configuration.headers.add("ConsistencyLevel", "eventual")

    response = await client.users.get(request_configuration=request_configuration)
    if response.value:
        user = response.value[0]
        print(f"{userid} found in AD. Principal: {user.user_principal_name}, "
                f"Id: {user.id}, Display: {user.display_name}")
        return {
                "display": user.display_name,
                "mail": user.mail,
                "id": user.id,
                "principal": user.user_principal_name
               }
    print(f"{userid} not found in AD")


asyncio.run(_get_user_status('clecoz'))

Thanks,
Cedric.

Activity

  1. moved this to Needs Triage 🔍 in Kiotaon Jan 30, 2025
  2. adam-blanchard commented on Jan 30, 2025

    @adam-blanchard

    Hi everyone.

    We're also facing this issue even with simple get user by id. Our data engineering pipelines started to fail today, and the only environment change I've identified is the upgrading of this dependency from 1.9.0 to 1.9.1.

    Same env setup as Cedric, and the code below is resulting in HTTPError: Valid url scheme and host required .

    I will try pinning the dependency to 1.9.0 as a workaround.

    import asyncio
    from azure.identity.aio import ClientSecretCredential
    from msgraph import GraphServiceClient
    
    TENANT_ID = '<REDACTED>'
    CLIENT_ID = '<REDACTED>'
    CLIENT_SECRET = '<REDACTED>'
    
    credential = ClientSecretCredential(TENANT_ID, CLIENT_ID, CLIENT_SECRET)
    scopes = ['https://graph.microsoft.com/.default']
    client = GraphServiceClient(credentials=credential, scopes=scopes)
    
    user = await client.users.by_user_id('<REDACTED>').get()
    HTTPError: Valid url scheme and host required
    File <command-3714762163119403>, line 1
    ----> 1 user = await client.users.by_user_id('<REDACTED>').get()
          2 print(user)
    File /local_disk0/.ephemeral_nfs/envs/pythonEnv-333e6112-c98d-42d6-a5c5-76f6b0493c1c/lib/python3.11/site-packages/msgraph/generated/users/item/user_item_request_builder.py:160, in UserItemRequestBuilder.get(self, request_configuration)
        157     raise Exception("Http core is null") 
        158 from ...models.user import User
    --> 160 return await self.request_adapter.send_async(request_info, User, error_mapping)
    File /local_disk0/.ephemeral_nfs/envs/pythonEnv-333e6112-c98d-42d6-a5c5-76f6b0493c1c/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:186, in HttpxRequestAdapter.send_async(self, request_info, parsable_factory, error_map)
        183     parent_span.record_exception(REQUEST_IS_NULL)
        184     raise REQUEST_IS_NULL
    --> 186 response = await self.get_http_response_message(request_info, parent_span)
        188 response_handler = self.get_response_handler(request_info)
        189 if response_handler:
    File /local_disk0/.ephemeral_nfs/envs/pythonEnv-333e6112-c98d-42d6-a5c5-76f6b0493c1c/lib/python3.11/site-packages/kiota_http/httpx_request_adapter.py:595, in HttpxRequestAdapter.get_http_response_message(self, request_info, parent_span, claims)
        592 if claims:
        593     additional_authentication_context[self.CLAIMS_KEY] = claims
    --> 595 await self._authentication_provider.authenticate_request(
        596     request_info, additional_authentication_context
        597 )
        599 request = self.get_request_from_request_information(
        600     request_info, _get_http_resp_span, parent_span
        601 )
        602 resp = await self._http_client.send(request)
    File /local_disk0/.ephemeral_nfs/envs/pythonEnv-333e6112-c98d-42d6-a5c5-76f6b0493c1c/lib/python3.11/site-packages/kiota_abstractions/authentication/base_bearer_token_authentication_provider.py:50, in BaseBearerTokenAuthenticationProvider.authenticate_request(self, request, additional_authentication_context)
         47     request.headers = HeadersCollection()
         49 if not request.headers.contains(self.AUTHORIZATION_HEADER):
    ---> 50     token = await self.access_token_provider.get_authorization_token(
         51         request.url, additional_authentication_context
         52     )
         53     if token:
         54         request.headers.add(f'{self.AUTHORIZATION_HEADER}', f'Bearer {token}')
    File /local_disk0/.ephemeral_nfs/envs/pythonEnv-333e6112-c98d-42d6-a5c5-76f6b0493c1c/lib/python3.11/site-packages/kiota_authentication_azure/azure_identity_access_token_provider.py:78, in AzureIdentityAccessTokenProvider.get_authorization_token(self, uri, additional_authentication_context)
         76     exc = HTTPError("Valid url scheme and host required")
         77     span.record_exception(exc)
    ---> 78     raise exc
         80 if parsed_url.scheme != "https" and (parsed_url.hostname not in self.LOCALHOST_STRINGS):
         81     span.set_attribute(self.IS_VALID_URL, False)
    
  3. adam-blanchard commented on Jan 30, 2025

    @adam-blanchard

    I have just done some more digging and pinning the package microsoft-kiota-http to version 1.9.0 has resolved the issue. Looks like microsoft-kiota-http 1.9.1 introduced the bug.

  4. adam-blanchard commented on Jan 30, 2025

    @adam-blanchard

    Confirmed with my team that pinning microsoft-kiota-http to 1.9.0 is a functioning workaround.

  5. lucas-bremond commented on Jan 31, 2025

    @lucas-bremond

    On a similar vibe, from 1.9.1, the following

    from azure.identity.aio import DefaultAzureCredential
    from msgraph_core import AzureIdentityAuthenticationProvider
    
    AzureIdentityAuthenticationProvider(
        credentials=DefaultAzureCredential(),
    )

    now raises a ValueError: Allowed host value cannot contain 'https://' or 'http://' prefix.

    Freezing microsoft-kiota-abstractions==1.9.0 solves the issue...

  6. moved this from Needs Triage 🔍 to Proposed 💡 in Kiotaon Feb 6, 2025
  7. moved this from Proposed 💡 to New📃 in Kiotaon Feb 6, 2025
  8. andrueastman commented on Feb 7, 2025

    @andrueastman
    Contributor

    Hey everyone,
    After investigations, this issue is related to microsoftgraph/msgraph-sdk-python-core#818 together with changes made in #459.

    In python 3.11, the formating of enum values to strings does not result in the enum value being set but the property name being used. This resulted in the httpclient.baseurl being set to something like NationalClouds.Global/APIVersion.v1/ rather than https://graph.microsoft.com/v1.0/. This change was unnoticed/inconsequenstial till the fix made in #459 that made the adapter use the base_url from the http client.

    This issue was fortunately already been resolved in microsoftgraph/msgraph-sdk-python-core#820 and therefore the solution here it to ensure that your dependency tree has msgraph-core version 1.2.1 or greater.

    https://github.com/microsoftgraph/msgraph-sdk-python-core/releases/tag/v1.2.1

  9. moved this from New📃 to In Progress 🚧 in Kiotaon Feb 10, 2025
  10. moved this from In Progress 🚧 to Done ✔️ in Kiotaon Feb 17, 2025
  11. cedriclecoz commented on Feb 17, 2025

    @cedriclecoz
    Author

    Sorry missed your previous comment, tested OK by switching back to my un-pinned requirement file.

    ie. kiota* has been bumped to 1.9.2 and msgraph-core to 1.3.1.

    Thanks,
    Cedric,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

  • Status
    Done ✔️

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions