PnP-Sites-Core icon indicating copy to clipboard operation
PnP-Sites-Core copied to clipboard

Apply template with pnp:header or pnp:footer gives error

Open waaromikniet opened this issue 4 years ago • 26 comments

Environment

[X] Office 365 / SharePoint Online

Expected or Desired Behavior

I want to apply this template using Pnp-PowerShell so I am using Apply-PnPProvisioningTemplate

<?xml version="1.0"?>
<pnp:Provisioning xmlns:pnp="http://schemas.dev.office.com/PnP/2020/02/ProvisioningSchema">
  <pnp:Preferences Generator="OfficeDevPnP.Core, Version=3.24.2008.0, Culture=neutral, PublicKeyToken=5e633289e95c321a" />
  <pnp:Templates ID="CONTAINER-TEMPLATE-B0667CBD6442459BB3D1A8760B13DC80">
    <pnp:ProvisioningTemplate ID="TEMPLATE-B0667CBD6442459BB3D1A8760B13DC80" Version="1" BaseSiteTemplate="SITEPAGEPUBLISHING#0" Scope="RootSite">
      <pnp:Header Layout="Compact" MenuStyle="MegaMenu" />
    </pnp:ProvisioningTemplate>
  </pnp:Templates>
</pnp:Provisioning>

I want my header to be set to compact

Observed Behavior

I am getting the following error'

Apply-PnPProvisioningTemplate : AADSTS65001: The user or administrator has not consented to use the application with ID '31359c7f-bd7e-475c-86db-fdb8c937548e' named 'PnP Management Shell'. Send an interactive authorization request for thi s user and resource. Trace ID: 69ec10b7-0a49-4039-bb8d-a52cd4bf6e00 Correlation ID: f5197f26-24fc-468d-bc87-d2ced9af9fe9 Timestamp: 2020-08-12 14:35:22Z

In my Azure AD I see a interrupted state for application: PnP Management Shell But that is a non existing app I can't find in the company apps list.

Steps to Reproduce

Install version 3.24.2008.0 of Pnp-PowerShell Use credential manager with a username and password to connect Apply my template example on a modern site collection (communication or group site)

Seems almost the same #2537

Same applies when using pnp:Footer

waaromikniet avatar Aug 12 '20 14:08 waaromikniet

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

ghost avatar Aug 12 '20 14:08 ghost

When I use Fiddler I do see a failed request passing by with status 400 Removed username and password

Request

POST https://login.microsoftonline.com/organizations/oauth2/v2.0/token HTTP/1.1
x-client-SKU: MSAL.Desktop
x-client-Ver: 4.17.0.0
x-client-CPU: x64
x-client-OS: Microsoft Windows NT 10.0.19041.0
x-client-current-telemetry: 2|1003,0|
x-client-last-telemetry: 2|0|1003,a1534677-5a48-4500-ae94-4b917072e2fd|invalid_grant|
x-ms-PKeyAuth: 1.0
x-ms-lib-capability: retry-after, h429
client-request-id: 56efa20f-51b6-408b-8f18-24baca11d8e9
return-client-request-id: true
x-app-name: UnknownClient
x-app-ver: 0.0.0.0
Accept: application/json
Content-Type: application/x-www-form-urlencoded
Host: login.microsoftonline.com
Cookie: fpc=Agd5Y4IlZ2BFr_B9e4KOvn8f_zjHAQAAAH4ExtYOAAAA; x-ms-gateway-slice=prod; stsservicecookie=ests; esctx=AQABAAAAAAAGV_bv21oQQ4ROqh0_1-tA97IhrV1G9JWaY6T2rkKC_Lq2eKTun7YmSZIoNvjJfeSBtOOFq85KgCET7n17hFf-qBsIIuHBtpRdK7uKXL9y-feGSI9hojfj_QR0JbWE_LKtHjmfh-tI83daL0vjWWf5qyGN3sQfVR_WC4YLZjIuQDhNUIldMiZx9L31y69DUUggAA
Content-Length: 248
Expect: 100-continue

client_id=31359c7f-bd7e-475c-86db-fdb8c937548e&client_info=1&scope=openid+offline_access+profile+https%3A%2F%2Fuser.sharepoint.com%2F%2F.default&grant_type=password&username=user%40user.onmicrosoft.com&password=%40************

Response

HTTP/1.1 400 Bad Request
Cache-Control: no-store, no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
client-request-id: 56efa20f-51b6-408b-8f18-24baca11d8e9
x-ms-request-id: f89e4f4d-7a51-4e40-9dd7-ea919d391802
x-ms-ests-server: 2.1.10922.14 - DUB2 ProdSlices
x-ms-clitelem: 1,65001,0,,
Set-Cookie: fpc=Agd5Y4IlZ2BFr_B9e4KOvn8f_zjHAQAAAKoFxtYOAAAA; expires=Fri, 11-Sep-2020 15:49:29 GMT; path=/; secure; HttpOnly; SameSite=None
Set-Cookie: x-ms-gateway-slice=prod; path=/; secure; httponly
Set-Cookie: stsservicecookie=ests; path=/; secure; samesite=none; httponly
Date: Wed, 12 Aug 2020 15:49:29 GMT
Content-Length: 608

{"error":"invalid_grant","error_description":"AADSTS65001: The user or administrator has not consented to use the application with ID '31359c7f-bd7e-475c-86db-fdb8c937548e' named 'PnP Management Shell'. Send an interactive authorization request for this user and resource.\r\nTrace ID: f89e4f4d-7a51-4e40-9dd7-ea919d391802\r\nCorrelation ID: 56efa20f-51b6-408b-8f18-24baca11d8e9\r\nTimestamp: 2020-08-12 15:49:29Z","error_codes":[65001],"timestamp":"2020-08-12 15:49:29Z","trace_id":"f89e4f4d-7a51-4e40-9dd7-ea919d391802","correlation_id":"56efa20f-51b6-408b-8f18-24baca11d8e9","suberror":"consent_required"}

When I use Connect-PnpOnline -UseWebLogin I log in via the browser. Then I get no error but template is just not working

waaromikniet avatar Aug 12 '20 18:08 waaromikniet

Look like it could be caused by the way the value is set in header object handler (ObjectSiteHeaderSettings.cs) by adding the new /_api/web/SetChromeOptions added by the commit from @PaoloPia

commit

waaromikniet avatar Aug 12 '20 19:08 waaromikniet

@waaromikniet i am also facing same issue with apply-template when header is included in the PnP template, is there any work around ? It is failing when I have Header included in the template, however once received it is working fine but due to not having header, Styles and Site assets like logo are not working ..could you please suggest any workaround that can be applied to by pass this issue and still get Styles and Site assets like logo applied ? Appreciate your help.

MMalikKhan avatar Aug 13 '20 01:08 MMalikKhan

@waaromikniet i removed <pnp:Header Layout="Standard" /> tag from the template and added Get-PnPProvisioningTemplate -Out $filename -PersistBrandingFiles:$true and logo and styles are there, I am not sure what i am loosing by not apply Header Layout, I guess my case both sites are created from the same template so maybe at the moment i can live without having header in that template until this bug is fixed...

MMalikKhan avatar Aug 13 '20 01:08 MMalikKhan

Same issue on my side. If I have Herader or Footer in the template the apply temaplte fails. Also I see another problem when applying <pnp:Theme ... with the template. I suppose they are related issues.

nemortu avatar Aug 13 '20 07:08 nemortu

@waaromikniet i am also facing same issue with apply-template when header is included in the PnP template, is there any work around ? It is failing when I have Header included in the template, however once received it is working fine but due to not having header, Styles and Site assets like logo are not working ..could you please suggest any workaround that can be applied to by pass this issue and still get Styles and Site assets like logo applied ? Appreciate your help.

You could remove the header if you don't need it. But if you need it:

If you use the PnP-PowerShell module, the only way is to go back to a release before June 9.

If you use PnP-SItes-Core in your custom tooling you could pull the code and make a temporary change in ObjectSiteHeaderSettings.cs. You could remove this section

                if (PnPProvisioningContext.Current != null)
                {
                    // Get an Access Token for the SetChromeOptions request
                    var spoResourceUri = new Uri(web.Url).Authority;
                    var accessToken = PnPProvisioningContext.Current.AcquireToken(spoResourceUri, null);

                    if (accessToken != null)
                    {
                        // Prepare the JSON request for SetChromeOptions
                        var jsonRequest = new
                        {
                            headerLayout = web.HeaderLayout,
                            headerEmphasis = web.HeaderEmphasis,
                            megaMenuEnabled = web.MegaMenuEnabled,
                        };

                        // Build the URL of the SetChromeOptions API
                        var setChromeOptionsApiUrl = $"{web.Url}/_api/web/SetChromeOptions";

                        // Make the POST request to the SetChromeOptions API
                        // and fail in case of any exception
                        HttpHelper.MakePostRequest(setChromeOptionsApiUrl,
                            jsonRequest,
                            "application/json",
                            accessToken);
                    }
                }
                else
                {
                    web.Update();
                    web.Context.ExecuteQueryRetry();
                }

and replace with

                    web.Update();
                    web.Context.ExecuteQueryRetry();

Maybe @PaoloPia or @erwinvanhunen can explain if we need to do some change on tenant level to make this code calling "/_api/web/SetChromeOptions" to work if you use username/password to authenticate because the code seems to fail when you do. I see this check

PnPProvisioningContext.Current != null

When this is false code reverts to the old way. I can't pin point what is happening here. I am using the company administrator account to test provisioning. I see checks like

    public static bool IsCurrentUserTenantAdmin(ClientContext clientContext)
    {
        if (PnPProvisioningContext.Current != null)
        {
            return IsCurrentUserTenantAdminViaGraph();
        }
        else
        {
            return IsCurrentUserTenantAdminViaSPO(clientContext);
        }
    }

Maybe using company administrator is not supported anymore using username/password? I also tested with an account that is only SharePoint administrator but then the same error.

waaromikniet avatar Aug 13 '20 08:08 waaromikniet

@waaromikniet : for my understanding: do you have legacy auth set to false in your tenant settings? If so, PnP PowerShell will use the PnP Management Shell multi-tenant application (if you provided consent to it before you'll find it in the enterprise applications list of your Azure AD, either by the name PnP Management Shell or can be called PnP Office 365 Management Shell). Do other PnP PowerShell cmdlets work as expected?

erwinvanhunen avatar Aug 13 '20 12:08 erwinvanhunen

@erwinvanhunen Were can I find this setting? You mean this:

image

This is EO and Office. NOt for mobile and desktop app?

I am testing on an out of the box E5 developer tenant that I just created. Other PnP-PowerShell commands work as expected. Only when adding pnp:Header or pnp:Footer to my template I get this error. I think because of the code added by @PaoloPia shown above.

I don't see any Enterprise applications with the name PNP in my Azure AD.

image

In my azure AD I do see this

image

image

In usage and insights I see this

image

For the consent. How can I consent when I am using PowerShell? I won't get the consent browser screen when not in interactive mode.

waaromikniet avatar Aug 13 '20 12:08 waaromikniet

I have exact same problem when trying to apply SiteFooter xml template to site. Problematic version is SharePointPnPPowerShellOnline 3.24.2008.0.

On my other computer I have version 3.19.2003.0 and xml template applies with no problems. Using same account on both computers.

ariv8 avatar Aug 13 '20 13:08 ariv8

@waaromikniet You can find the setting by executing Get-SPOTenant or Get-PnPTenant. The setting is called LegacyAuthProtocolsEnabled. If set to false it means you're not using classic auth anymore and we will have to fallback to OAuth behind the scenes for authentication. For that it uses the PnP Management Shell multi-tenant app.

The easiest way to provide consent is by doing:

Connect-PnPOnline -PnPManagementShell -LaunchBrowser -Url https://yourtenant.sharepoint.com

In your case, as the app is listed as 'interrupted', you might want to remove the entry from your enterprise applications first (I have not seen an 'interrupted' state until now before)

erwinvanhunen avatar Aug 13 '20 14:08 erwinvanhunen

Added info: setting LegacyAuthProtocolsEnabled to true or false is not instant. That can take a few hours.

erwinvanhunen avatar Aug 13 '20 14:08 erwinvanhunen

@erwinvanhunen
LegacyAuthProtocolsEnabled is set to true in my tenant. I have created another tenant and it has the same issue. In that tenant LegacyAuthProtocolsEnabled is also set to true.

I can't remove the entry from enterprise applications because it is not present in the list.

It looks like the code for the header or footer is trying to use oAuth even when in legacy authentication?

waaromikniet avatar Aug 13 '20 14:08 waaromikniet

right... okay, so for pure auth, PnP PowerShell will use the legacy auth (SharePointOnlineCredentials) option then. I will check what's exactly happening in the engine when it comes to the headers and footers. There are scenarios where we have to fallback to the Graph or other REST calls and there we need an access token. We utilize a fallback to use the PnP Management Shell OAuth for that. I know for a fact that for the footer we make REST calls, but those are directly towards SharePoint, so there we can utilize the built in ways to acquire an access token for SharePoint (cookie based, etc.), but I'll have to dive a bit deeper into this specific case to see what's going on.

erwinvanhunen avatar Aug 13 '20 15:08 erwinvanhunen

Have you tried the Connect-PnPOnline -PnPManagementShell -Url option? It will go through the consent flow and add the enterprise app to your AD. You can, after you have consented and not want to have the entry there always remove it again from your AD.

erwinvanhunen avatar Aug 13 '20 15:08 erwinvanhunen

right... okay, so for pure auth, PnP PowerShell will use the legacy auth (SharePointOnlineCredentials) option then. I will check what's exactly happening in the engine when it comes to the headers and footers. There are scenarios where we have to fallback to the Graph or other REST calls and there we need an access token. We utilize a fallback to use the PnP Management Shell OAuth for that. I know for a fact that for the footer we make REST calls, but those are directly towards SharePoint, so there we can utilize the built in ways to acquire an access token for SharePoint (cookie based, etc.), but I'll have to dive a bit deeper into this specific case to see what's going on.

@erwinvanhunen When I look at the footer code I see the same kind of request being done to /_api/web/SetChromeOptions like in the header code. When I use a footer instead of a header in my template I get the same error. Reverting back to older version of PnP-PowerShell solves the problem. Like mentioned by @ariv8

waaromikniet avatar Aug 13 '20 15:08 waaromikniet

Have you tried the Connect-PnPOnline -PnPManagementShell -Url option? It will go through the consent flow and add the enterprise app to your AD. You can, after you have consented and not want to have the entry there always remove it again from your AD.

@erwinvanhunen When I perform this and give the app the consent and run my code again I am no longer getting the error. For header or footer. So it seems that LegacyAuthProtocolsEnabled is ignored when setting header of footer and if you haven't given the consent the code will fail.

waaromikniet avatar Aug 13 '20 16:08 waaromikniet

Right. So a change has happened that require this consent to be in place. I'll take it up with the team.

erwinvanhunen avatar Aug 13 '20 16:08 erwinvanhunen

@erwinvanhunen

I did some more testing an indeed when you remove the enterprise app the error gets back. I did notice 1 strange thing during testing that I would like to mention to you for you or your team when testing.

I use these 2 lines to apply my template

Connect-PnPOnline -Url https://myurl
Apply-PnPProvisioningTemplate -Path C:\Temp\Export\Template.xml

When I put these 2 lines inside a .ps1 file and run the file I get the error on screen.

Apply-PnPProvisioningTemplate : AADSTS65001: The user or administrator has not consented to use the application with ID
'31359c7f-bd7e-475c-86db-fdb8c937548e' named 'PnP Management Shell'. Send an interactive authorization request for thi
s user and resource.
Trace ID: 69ec10b7-0a49-4039-bb8d-a52cd4bf6e00
Correlation ID: f5197f26-24fc-468d-bc87-d2ced9af9fe9
Timestamp: 2020-08-12 14:35:22Z

When I open a new PowerShell window and manually type these 2 commands the template doesn't get applied but I didn't get the error or any other feedback on screen that the template fails. So when manually typing the command you could get the idea that everything went fine but actually the template fails. We had this problem and thought the template worked but after a week we noticed that nothing had happend on our site. When creating a test .ps1 file we finally noticed the error.

waaromikniet avatar Aug 14 '20 07:08 waaromikniet

We have identified the issue and have just merged a solution. We are discussing internally if we should release an intermediate build.

erwinvanhunen avatar Aug 14 '20 12:08 erwinvanhunen

@erwinvanhunen, is this change present in the august release for PnP PowerShell?

waaromikniet avatar Aug 26 '20 12:08 waaromikniet

@erwinvanhunen I'm getting similar issues would you be able to provide an update on this?

jaywellings avatar Sep 03 '20 15:09 jaywellings

@erwinvanhunen could you please release to correct this issue ? there is an issue when my template contains pnp:header or pnp:footer tag. This error impacts an important project in production. Thanks in advance

colletolivier avatar Sep 22 '20 12:09 colletolivier

We have narrowed down the issue to a change in the provisioning engine that we unfortunately had to implement. The -current- release of PnP PowerShell and PnP Sites Core will work, however, you have to perform 2 specific steps (only one time):

Connect-PnPOnline -Url <yoursite> -PnPManagementShell

Authenticate and go through the consent approval, if required (if you have done this before you will not be prompted for consent).

Then:

Connect-PnPOnline -Graph

Again authenticate and provide consent.

This will put the right consent in place that the provisioning engine requires to provision the header and footer (and other artifacts that require OAuth based authentication).

In the upcoming release of PnP PowerShell we will provide a single cmdlet, Register-PnPPowerShellAccess that will talk you through the consent steps required for both resources (SharePoint and The Graph).

erwinvanhunen avatar Sep 22 '20 14:09 erwinvanhunen

Hi Erwin, thanks a lot for your response. I will test and get back to you

Best regards,

----- Mail original ----- De: "Erwin van Hunen" [email protected] À: "pnp/PnP-Sites-Core" [email protected] Cc: "colletolivier" [email protected], "Comment" [email protected] Envoyé: Mardi 22 Septembre 2020 16:06:55 Objet: Re: [pnp/PnP-Sites-Core] Apply template with pnp:header or pnp:footer gives error (#2732)

We have narrowed down the issue to a change in the provisioning engine that we unfortunately had to implement. The -current- release of PnP PowerShell and PnP Sites Core will work, however, you have to perform 2 specific steps (only one time): Connect-PnPOnline - Url < yoursite > - PnPManagementShell

Authenticate and go through the consent approval, if required (if you have done this before you will not be prompted for consent).

Then: Connect-PnPOnline - Graph

Again authenticate and provide consent.

This will put the right consent in place that the provisioning engine requires to provision the header and footer (and other artifacts that require OAuth based authentication).

In the upcoming release of PnP PowerShell we will provide a single cmdlet, Register-PnPPowerShellAccess that will talk you through the consent steps required for both resources (SharePoint and The Graph).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub , or unsubscribe .

colletolivier avatar Sep 22 '20 20:09 colletolivier

Hi, I Have executed the two command (authentication and consent) but i have always the following isssue :

Apply-PnPProvisioningTemplate : {"odata.error":{"code":"-2147024891, System.UnauthorizedAccessException","message":{"lang":"fr-FR","value":"Acc\u00e8s refus\u00e9. Vous n\u2019\u00eates pas autoris\u00e9 \u00e0 effectuer cette action ou \u00e0 acc\u00e9der \u00e0 cette ressource."}}} At C:\Workspaces\KEOLIS\KEOLIS - KAPRAO\KAPRAO_ProjectSiteProvisioning\Engine\Templates\Projetlys\projet\debug.ps1:23 char:1

  • Apply-PnPProvisioningTemplate -Path "projet-debug.xml" -Parameters @{ ...

  • CategoryInfo : WriteError: (:) [Apply-PnPProvisioningTemplate], Exception

  • FullyQualifiedErrorId : EXCEPTION,PnP.PowerShell.Commands.Provisioning.Site.ApplyProvisioningTemplate

In attachment, the used template and the powershell script executed.

Thanks Olivier

----- Mail original -----

De: "Erwin van Hunen" [email protected] À: "pnp/PnP-Sites-Core" [email protected] Cc: "colletolivier" [email protected], "Comment" [email protected] Envoyé: Mardi 22 Septembre 2020 16:06:55 Objet: Re: [pnp/PnP-Sites-Core] Apply template with pnp:header or pnp:footer gives error (#2732)

We have narrowed down the issue to a change in the provisioning engine that we unfortunately had to implement. The -current- release of PnP PowerShell and PnP Sites Core will work, however, you have to perform 2 specific steps (only one time): Connect-PnPOnline - Url < yoursite > - PnPManagementShell

Authenticate and go through the consent approval, if required (if you have done this before you will not be prompted for consent).

Then: Connect-PnPOnline - Graph

Again authenticate and provide consent.

This will put the right consent in place that the provisioning engine requires to provision the header and footer (and other artifacts that require OAuth based authentication).

In the upcoming release of PnP PowerShell we will provide a single cmdlet, Register-PnPPowerShellAccess that will talk you through the consent steps required for both resources (SharePoint and The Graph).

— You are receiving this because you commented. Reply to this email directly, view it on GitHub , or unsubscribe .

colletolivier avatar Sep 29 '20 20:09 colletolivier