Azure
Unexpected return type <class 'str'> from ContentDecodePolicy.deserialize_from_http_generics
Used versions:
Azurite: 3.32.0 Python: 3.11 azure-storage-blob: 12.23.1
I am running azurite locally:
azurite-blob --silent --location c:\foo\bar\azurite --debug c:\foo\bar\azurite\debug.log
Then, consider the following Python script with the connection string from the docs here:
from azure.storage.blob import ContainerClient
CONNECTION_STRING = "DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;"
container_client = ContainerClient.from_connection_string(
conn_str=CONNECTION_STRING,
container_name="mycontainer",
)
container_client.create_container()
Here, the create_container() call fails with:
Unexpected return type <class 'str'> from ContentDecodePolicy.deserialize_from_http_generics.
Traceback (most recent call last):
Cell In[43], line 1
runfile('D:/azure-blob-test.py', wdir='D:')
File C:\PR\Anaconda3\envs\adhoc\Lib\site-packages\debugpy\_vendored\pydevd\_pydev_bundle\pydev_umd.py:175 in runfile
execfile(filename, namespace)
File C:\PR\Anaconda3\envs\adhoc\Lib\site-packages\debugpy\_vendored\pydevd\_pydev_bundle\_pydev_execfile.py:14 in execfile
exec(compile(contents + "\n", file, 'exec'), glob, loc)
File D:/azure-blob-test.py:10
container_client.create_container()
File C:\PR\Anaconda3\envs\adhoc\Lib\site-packages\azure\core\tracing\decorator.py:94 in wrapper_use_tracer
return func(*args, **kwargs)
File C:\PR\Anaconda3\envs\adhoc\Lib\site-packages\azure\storage\blob\_container_client.py:321 in create_container
process_storage_error(error)
File C:\PR\Anaconda3\envs\adhoc\Lib\site-packages\azure\storage\blob\_shared\response_handlers.py:186 in process_storage_error
exec("raise error from None") # pylint: disable=exec-used # nosec
File <string>:1
HttpResponseError: <style>
html,body,div,span,object,iframe,h1,h2,h3,h4,h5,h6,p,blockquote,pre,abbr,address,cite,code,del,dfn,em,img,ins,kbd,q,samp,small,strong,sub,sup,var,b,i,dl,dt,dd,ol,ul,li,fieldset,form,label,legend,table,caption,tbody,tfoot,thead,tr,th,td,article,aside,canvas,details,figcaption,figure,footer,header,hgroup,menu,nav,section,summary,time,mark,audio,video{margin:0;padding:0;border:0;outline:0;font-size:100%;vertical-align:baseline;background:transparent} body{line-height:1} article,aside,details,figcaption,figure,footer,header,hgroup,menu,nav,section{display:block} nav ul{list-style:none} blockquote,q{quotes:none} blockquote:before,blockquote:after,q:before,q:after{content:none} a{margin:0;padding:0;font-size:100%;vertical-align:baseline;background:transparent} ins{background-color:#ff9;color:#000;text-decoration:none} mark{background-color:#ff9;color:#000;font-style:italic;font-weight:bold} del{text-decoration:line-through} abbr[title],dfn[title]{border-bottom:1px dotted;cursor:help} table{border-collapse:collapse;border-spacing:0} hr{display:block;height:1px;border:0;border-top:1px solid #ccc;margin:1em 0;padding:0} input,select{vertical-align:middle}
*{cursor:default;} html,body{margin:0;padding:0;height:100%;text-align:center;background:#ddd;font-weight:normal;font-size:15px;line-height:1.5;font-family:Arial;}
body{margin:0;padding:0;} #header{border-bottom:0px solid #0080c6;color:#fff;font-size:2em;line-height:1.25em;font-weight:bold;padding:0;text-shadow:1px 1px 0 rgba(0,0,0,0.5);height:95px;}
.wrapper{width:800px;margin:0 auto;padding:0;position:relative;text-align:left; background:#F27022; height:95px;} .hl{float:left;padding:15px;} .hr{float:right;} table{border:0px;border-collapse:collapse;width:100%;vertical-align:top;background:#fff;}
a{color:#0081c5;text-decoration:none;border-bottom:1px dotted #0080c6;cursor:pointer;font-weight:bold} a:hover{color:#e62859;text-decoration:none;border-bottom:1px dotted #e62859;} a:active,a:focus{outline:none;} a img{border:none;}
.tdl{width:200px;font-weight:
This is not specific to create_container, but many other methods fail with essentially the same error.
What's the issue and how to go around it? Thanks!
@jplauri
Would you please share the debug log of Azurite, then we can continue the investigation? We are not expert of Python SDK, so not family with the python failure. From the python error you shared, we can't find any clue of the request you send to Azurite and Azurite responds. The Azurite debug log can help to identify if any issue on Azurite side.
Besides that, if you change the connectionstring point to a product Azure storage account, will you code work in same environment?
It seems I never get that far as to get anything interesting into the logs:
2024-10-22T05:28:48.445Z info: Azurite Blob service is starting on 127.0.0.1:10000
2024-10-22T05:28:48.447Z info: AccountDataStore:init() Refresh accounts from environment variable AZURITE_ACCOUNTS with value undefined
2024-10-22T05:28:48.447Z info: AccountDataStore:init() Fallback to default emulator account devstoreaccount1.
2024-10-22T05:28:48.493Z info: BlobGCManager:start() Starting BlobGCManager. Set status to Initializing.
2024-10-22T05:28:48.494Z info: BlobGCManager:start() Trigger mark and sweep loop. Set status to Running.
2024-10-22T05:28:48.494Z info: BlobGCManager:markSweepLoop() Start next mark and sweep.
2024-10-22T05:28:48.494Z info: BlobGCManager:markSweep() Get all extents.
2024-10-22T05:28:48.494Z info: BlobGCManager:start() BlobGCManager successfully started.
2024-10-22T05:28:48.508Z info: BlobGCManager:markSweep() Got 0 extents.
2024-10-22T05:28:48.508Z info: BlobGCManager:markSweep() Get referred extents.
2024-10-22T05:28:48.509Z info: BlobGCManager:markSweep() Got referred extents, unreferenced extents count is 0.
2024-10-22T05:28:48.509Z info: BlobGCManager:markSweepLoop() Mark and sweep finished, taken 15ms.
2024-10-22T05:28:48.509Z info: BlobGCManager:markSweepLoop() Sleep for 600000ms.
2024-10-22T05:28:48.510Z info: Azurite Blob service successfully listens on http://127.0.0.1:10000
It works perfectly fine in a real environment, i.e., when the only difference is the connection string.
@jplauri
It looks Azurite not receive any request. It might be a client issue that error happen before send a request to Azurite.
It looks Azurite not receive any request. It might be a client issue that error happen before send a request to Azurite.
That makes sense. But given that I am using the connection string from the Azurite docs, what could be the issue? Put differently, I wonder if my snippet works for anyone else.
@jplauri It looks Azurite not receive any request. It might be a client issue that error happen before send a request to Azurite.
That makes sense. But given that I am using the connection string from the Azurite docs, what could be the issue? Put differently, I wonder if my snippet works for anyone else.
We are not python expert, so not sure what's wrong from python error. And there are no request send to Azurite, so the error is not from Azurite. You might can first raise a issue to get help from python sdk team (https://github.com/Azure/azure-sdk-for-python/issues) and make sure your program works with product Azure. Then if it doesn't work with Azurite only, we will look into it.
@jplauri It looks Azurite not receive any request. It might be a client issue that error happen before send a request to Azurite.
That makes sense. But given that I am using the connection string from the Azurite docs, what could be the issue? Put differently, I wonder if my snippet works for anyone else.
We are not python expert, so not sure what's wrong from python error. And there are no request send to Azurite, so the error is not from Azurite. You might can first raise a issue to get help from python sdk team (https://github.com/Azure/azure-sdk-for-python/issues) and make sure your program works with product Azure. Then if it doesn't work with Azurite only, we will look into it.
Sorry for being unclear - I can already confirm that the snippet works perfectly fine with product Azure and therefore the SDK team is unlikely to be able to help.
I can only imagine this being an issue with the Azurite-provided connection string and/or my specific environment. Would you please be able to look into this?
@jplauri
Would you please check if you can connect to your local Azurite with storage emulator? https://learn.microsoft.com/en-us/azure/storage/storage-explorer/vs-azure-tools-storage-manage-with-storage-explorer?tabs=windows#local-storage-emulator
BTW, I just tested you program on my machine, and it works well with Azurite. It looks might be some issue on your local environment.
Would you like to remove "--silent" when start Azurite, and see what will be report in Azurite output and debug log? Does Azurite start successfully on you machine? Does the default blob port 10000 is occupied on your machine for Azurite?
It seems that the connection string is OK, and running this with azure-cli gives more descriptive output. The issue appears to be that localhost is blocked in my environment. A fix to do e.g., os.environ["NO_PROXY"] = "localhost,127.0.0.1".
Thus, there is little Azurite or the SDK team can do. Many thanks for your help and taking the time nevertheless!