Exporting Logic App Standard to VS Code
If you work on Standard logic app workflows using the Azure portal, you might find yourself wanting to use Visual Studio Code instead at some point. When you switch to Visual Studio Code and install the Azure Logic Apps (Standard) extension, you get the expanded benefits available only with the extension, for example: Better control over your development environment. Faster development experience with local debugging and testing. Integration with source control repositories. Automated parameterization for new and existing connections, which simplifies cross-environment deployment. Automated generation for deployment scripts, templates for Azure connectors, and deployment with CI/CD pipelines. Starting with the Logic Apps Standard extension 5.18.7, you will find a new way to create Logic Apps Standard workspaces from a package exported from Azure Portal. You can find the step-by-step process on our official documentation. Watch the video below to see how this feature works.
Introduction and experience of Logic App Standard Advanced Tools – Part I
Scenario Microsoft announced General Availability (GA) of Logic App standard on 25th May 2021. We created an intelligent and efficient tool to self-troubleshoot Azure Logic App Standard. This tool integrated several useful features for Logic App Standard which are not available in Logic App portal yet. This blog introduces how to use this tool to help manage and troubleshoot logic app standard. This tool is still under development and this article will introduce some features so that it is Part I. We will have Part II and Part III in the future. Thanks! References GitHub - Drac-Zhang/Logic-App-STD-Advanced-Tools Services Used Azure Logic Apps (Standard) – Need Kudo Permission Introduction 1) Download tool: 2) Install tool: Just drag it into Kudo and it will install automatically: Help Page If you want to read Help Page first, please use this command LogicAppAdvancedTool -? (Main page of Help Page) LogicAppAdvancedTool [Command] -? (Introduction of each command) LogicAppAdvancedTool [Command] [Sub Command]-? (Introduction of each sub command) For example: Command format Please use command LogicAppAdvancedTool [Command] LogicAppAdvancedTool [Command] -wf [WorkflowName] For some other commands, they have sub-commands, eg. SyncToLocal: LogicAppAdvancedTool [Command] [SubCommand] Please note: Commands are case-insensitive. Command reference Backup: Retrieve all the definitions which can be found in Storage Table and save as Json files. The storage table saves the definition for the past 90 days by default even they have been deleted. Usage: Backup [options] Options: -d|--date (Optional) Retrieve workflow definitions which be modified/created later than this date (format: "yyyyMMdd"). -? Show help information. For example: CancelRuns: Cancel all the running/waiting instances of a workflow. Please note: Be aware of this command will cause data loss. Usage: CancelRuns [options] Options: -wf|--workflow (Mandatory) Workflow Name. -? Show help information. For example: CleanJobQueue: (Deprecated) Clear Logic App storage queue, this action could cause data loss. CleanUpContainers: Delete all the Logic App auto-generated blob containers for run history before a specific date. Usage: CleanUpContainers [options] Options: -wf|--workflow (Optional) The name of workflow. If not provided, then all the workflow containers will be deleted. -d|--date Delete containers before this date (format: "yyyyMMdd"), UTC time. -? Show help information. For example: CleanUpTables: Delete all the Logic App auto-generated storage tables for run history before a specific date. Usage: CleanUpTables [options] Options: -wf|--workflow (Optional) The name of workflow. If not provided, then all the workflow containers will be deleted. -d|--date (Mandatory) Delete run history related tables before this date (format: "yyyyMMdd"), UTC time. -? Show help information. For example: CleanUpRunHistory: Combined command of CleanUpContainers and CleanUpTables. Usage: CleanUpRunHistory [options] Options: -wf|--workflow (Optional) The name of workflow. If not provided, then all the workflow containers will be deleted. -d|--date (Mandatory) Delete run history related resources before this date (format: "yyyyMMdd"), UTC time. -? Show help information. For example: Check connectivity: Check the connection between Logic App and Storage Account via DNS resolution and TCP ping of 443 port. This command needs Kudu site is available. Usage: CheckConnectivity [options] Options: -? Show help information. For example: Clone: Clone a workflow to a new workflow, only support for same Logic App and same kind (stateful or stateless). Usage: Clone [options] Options: -sn|--sourcename (Mandatory) Source Workflow Name. -tn|--targetname (Mandatory) Target Workflow Name. -v|--version (Optional) Version of the workflow the latest version will be cloned, if not provided the latest version will be selected.) -? Show help information. For example: ConvertToStateful: Clone a stateless workflow and create a new stateful workflow. Usage: ConvertToStateful [options] Options: -sn|--sourcename (Mandatory) Source Workflow Name (Stateless) -tn|--targetname (Mandatory) Target Workflow Name (Stateful) -? Show help information. For example: Decode: Decode a workflow based on provided version to human readable content. Usage: Decode [options] Options: -wf|--workflow (Mandatory) Workflow Name. -v|--version (Mandatory) Version, the first part of the backup file name. -? Show help information. For example: GenerateTablePrefix: Generate Logic App/Workflow's storage table prefix. Usage: GenerateTablePrefix [options] Options: -wf|--workflow (Optional) Workflow name, if not provided, only Logic App prefix will be generated). -? Show help information. For example: GenerateRunHistoryUrl: Generate run history of failure runs of a specific workflow on a specific day. The url can directly open run history page. Usage: GenerateRunHistoryUrl [options] Options: -wf|--workflow (Mandatory) The name of workflow. -d|--date (Mandatory) The date (format: "yyyyMMdd") you would like to retrieve logs, UTC time. -f|--filter (Optional) Filter for specific exception messages. -? Show help information. For example: IngestWorkflow: (In development) This is an experimental feature. NOT fully tested, DON'T use in PROD environment!!! Ingest a workflow into Storage Table directly to bypass workflow definition validation. ListVersions: List all the existing versions of a workflow. Usage: ListVersions [options] Options: -wf|--workflow (Mandatory) Workflow Name. -? Show help information. For example: ListWorkflows: List all the existing workflows which can be found in the storage table. Usage: ListWorkflows [options] Options: -? Show help information. For example: RestoreAll: Restore all the workflows which were deleted accidentally. Please note: Restore all workflows which have been deleted, the existing workflows will not be impacted. Usage: RestoreAll [options] Options: -? Show help information. For example: RestoreSingleWorkflow: Restore a workflow which has been deleted accidentally. Usage: RestoreSingleWorkflow [options] Options: -wf|--workflow (Mandatory) The name of the workflow. -? Show help information. For example: RestoreRunHistory: Restore run history of a deleted/overwritten workflow. Please note: This is an experimental feature that might cause unexpected behavior in Logic App runtime since we directly modify workflow id. Usage: RestoreRunHistory [options] Options: -wf|--workflow (Mandatory) Workflow name. -? Show help information. For example: RetrieveFailures: Retrieve all the detailed failure information of a workflow for a specific day/run. Usage: RetrieveFailures [command] [options] Options: -? Show help information. Commands: Run 'RetrieveFailures [command] -?' for more information about command. Date Retrieve all the detailed failure information of a workflow for a specific day. Run Retrieve all the detail failure information of a workflow for a specific run. For example: Revert: Revert a workflow to a specific version. Usage: Revert [options] Options: -wf|--workflow (Mandatory) Workflow Name. -v|--version (Mandatory) Version, the first part of the backup file name. -? Show help information. For example: SyncToLocal: Sync remote wwwroot folder of Logic App Standard to local project. This command must run on a local computer. There are 3 subcommands for different usage, please use '-?' for more information. Please note: Local computers need to have access to Storage Account. Usage: SyncToLocal [command] [options] Options: -? Show help information. Commands: Run 'SyncToLocal [command] -?' for more information about a command. Auto Auto mode, there's no prompt dialog and can be set as schedule task for regular execution. Batch Batch mode, read configuration file (JSON format) from local folder and sync all the Logic Apps which are provided in config without prompt confirmation dialog. Normal Normal mode for manual sync, provides prompt dialog for confirmation of each step. For example: SearchInHistory: Search for a keyword in workflow run history based on date. Usage: SearchInHistory [options] Options: -wf|--workflow (Mandatory) The name of workflow. -d|--date (Mandatory) Date (format: "yyyyMMdd") of the logs need to be searched, UTC time. -k|--keyword (Mandatory) The keyword you would like to search for. -b|--includeBlob (Optional) true/false, whether needs to include the run history which saved as blob. Only the blob size less than 1MB will be checked due to memory saving. -of|--onlyFailures (Optional) Whether only to search for failed runs. -? Show help information. For example:Automatic Regeneration of Azure Managed Connectors Connection keys in VS Code Extension
Starting with version 4.57.6, the Azure Logic Apps (Standard) extension for Visual Studio Code will automatically regenerate the connection keys required to allow the extension to access Azure Managed Connections.IBM MQ Built-in (In-App) connector and Queue Handles: The math behind Queue Handles
In some scenarios, we found that it might be challenging to understand how IBM MQ server needs to be configured to be able to poll without experiencing the dreaded IBM MQ returned Reason Code: 2017 - MQRC_HANDLE_NOT_AVAILABLE.How to use azure logic app to update AAD user’s password automatically
Scenario Azure logic app is an extraordinary cloud automation application. For updating Azure Active Directory user’s password in batches and automatically, azure logic app consumption or a logic app standard can invoke Azure Active Directory Graph API but it requires specific permissions. References passwordAuthenticationMethod: resetPassword - Microsoft Graph beta | Microsoft Learn Sign in with resource owner password credentials grant - Microsoft Entra | Microsoft Learn List passwordMethods - Microsoft Graph beta | Microsoft Learn Update user - Microsoft Graph v1.0 | Microsoft Learn Services Used Azure Logic App (Consumption or Standard) Azure Active Directory (AAD) Solution 1 1.Create an AAD application registration 2.Add permission: UserAuthenticationMethod.ReadWrite.All More details: https://learn.microsoft.com/en-us/graph/api/authenticationmethod-resetpassword?view=graph-rest-beta&tabs=http#permissions 3.Grant admin consent 4.Set up a logic app designer Here we selected 'When a http request is received' as a trigger. Action 1: HTTP – Get token This action is used to get token. This token will be used in the following actions. Method: POST URL: https://login.microsoftonline.com/{tenantID}/oauth2/v2.0/token Content-Type: application/x-www-form-urlencoded Body: client_id={MyClientID} &scope=https%3A%2F%2Fgraph.microsoft.com%2F.default &client_secret={MyClientSecret} &grant_type=password &username={MyUsername}%40{myTenant}.com &password={MyPassword} Reference: https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth-ropc Action 2: HTTP – Get Pwd ID This action is used to get Password Method ID. Method: GET URL: https://graph.microsoft.com/beta/me/authentication/passwordMethods Content-type: application/json Reference: https://learn.microsoft.com/en-us/graph/api/authentication-list-passwordmethods?view=graph-rest-beta&tabs=http Action 3: HTTP – Update Pwd This action is used to update the password of a user. Method: POST URL: https://graph.microsoft.com/beta/users/{userObjectId | userPrincipalName}/authentication/passwordMethods/{passwordMethodId}/resetPassword Content-type: application/json Body: { "newPassword": "{myNewPassword}" } Reference: https://learn.microsoft.com/en-us/graph/api/authenticationmethod-resetpassword?view=graph-rest-beta&tabs=http#http-request In URI, we can use this Expression to get the value of passwordMethodId: body('HTTP_2_-_Get_Pwd_ID')['value'][0]['id'] Solution 2 1.Grant 4 permissions to application registration and grant admin consent User.ManageIdentities.All User.EnableDisableAccount.All User.ReadWrite.All Directory.ReadWrite.All Reference: https://learn.microsoft.com/en-us/graph/api/user-update?view=graph-rest-1.0&tabs=http#permissions 2.Add role assignment ‘User Administrator’ to application registration In delegated access, the calling app must be assigned the Directory.AccessAsUser.All delegated permission on behalf of the signed-in user. In application-only access, the calling app must be assigned the User.ReadWrite.All application permission and at least the User Administrator Azure AD role. Reference: https://learn.microsoft.com/en-us/graph/api/user-update?view=graph-rest-1.0&tabs=http 3.Set up a logic app designer Here we also selected 'When a http request is received' as a trigger. Action 1: HTTP – Get token This action is used to get token. This token will be used in the following actions. Method: POST URL: https://login.microsoftonline.com/{tenantID}/oauth2/v2.0/token Content-type: application/x-www-form-urlencoded Body: client_id={MyClientID} &scope=https%3A%2F%2Fgraph.microsoft.com%2F.default &client_secret={MyClientSecret} &grant_type=client_credentials Action 2: HTTP – Update Pwd This action is used to update the password of a user. Method: PATCH URL: https://graph.microsoft.com/v1.0/users/{userObjectId} Content-type: application/json Body: { "passwordProfile": { "forceChangePasswordNextSignIn": false, "password": "{myNewPassword}" } } Reference: https://learn.microsoft.com/en-us/graph/api/user-update?view=graph-rest-1.0&tabs=http#example-3-update-the-passwordprofile-of-a-user-to-reset-their-password Result We can check user password update records on AAD audit logs on azure portal: AAD page -> Users -> AAD audit logs
Scaling Logic Apps Standard – Sustained Message Processing System
In the previous blog of this blog post series, we discussed how Logic App standard can be used to process high throughput event data at a sustained rate over long periods of time. In this blog, we will see how Logic App standard can be used to process high throughput message data that can facilitate the decoupling of applications and services. We simulate a real-life use case where messages are sent to a Service Bus queue at a sustained rate for processing, and we use a templated Logic App workflow to process the messages in real-time. The business logic in the templated workflow can be easily replaced by the customer to actions that encompass their unique processing of the relevant messaging information. To better showcase the message processing capabilities, we will discuss two scaling capabilities, one for vertical scaling (varying the performance of service plans), and another horizontal scaling (varying the number of service plan instances). Vertical scaling capabilities of the Logic App Standard with Built-In Service Bus Connector In this section, we will investigate the vertical scaling capabilities of the Logic App Service Bus connector, conducting experiments to find the maximum message throughput supported by each of the standard Logic App SKUs from WS1 to WS3. The workflow uses the Service Bus built-in trigger, so the messages are promptly picked up and are processed in the run at par with ingress rate. like the one shown below - available at our Template Gallery. Customers can replace the Business Logic and Compensation Logic to handle their business scenarios. For this investigation, we used the out-of-the-box Logic Apps Standard configuration for scaling: 1 always ready instance 20 maximum burst instances We also used the default trigger batch size of 50. Experiment Methodology For each experiment we selected one of the available SKUs (WS1, WS2, WS3), and supplied a steady influx of X messages per minute to the connected Service Bus queue in one experiment. We conduct multiple experiments for each SKU and gradually increase X until the Logic App cannot process all the messages immediately. For each experiment, we pushed enough (1 million) messages in total to the queue to ensure that each workflow reaches a steady state processing rate with its maximum scaling. Environment Configuration The experiment setup is summarized in the table below: Tests setup Single Stamp Logic App Number of workflows 1 Templated Triggers Service Bus Trigger batch size 50 Actions Service Bus, Scope, Condition, Compose Number of storage accounts 1 Prewarmed instances 1 Max scale settings 20 Message size 1 KB Service Bus queue max size 2 GB Service Bus queue message lock duration 5 minutes Service Bus queue message max delivery count 10 Experiment results We summarize the experiment results in the table below. If the default maximum scaling of 20 instances is adopted, then the throughput we measured here serves as a good reference for the upper bound of message processing powers: WS Plan Message Throughput Time to process 1M messages WS1 9000 messages/minute 120 minutes WS2 19000 messages/minute 60 minutes WS3 24000 messages/minute 50 minutes In all the experiments, the Logic App scaled out to 20 instances at steady state. 📝 Complex business logic, which requires more actions and/or longer processing times, can change those values. Findings Understand the scaling and bottlenecks In the vertical scaling experiments, we limited the maximum instance count to 20. Under this setting, we sometimes observe "dead-letter" messages being generated. With Service Bus, messages become "dead-letters" if they are not processed within the lock duration for all delivery attempts. This means that the workflow takes more than 5 minutes to complete the scope/business logic for some messages. The root cause is that the Service Bus trigger fetches messages faster than the workflow actions can process them. As we can see in the following figure, the Service Bus trigger can fetch as much as 60k messages per minute, but the workflow can only process less than 30k messages per minute. Recommendations We recommend going with the default scaling settings if your workload is well below the published message throughput and increase the maximum burst when a heavier workload is expected. Horizontal scaling capabilities of the Logic App Service Bus connector In this section, we probe into the horizontal scaling of Logic App message handling capabilities with varying instance counts. We conduct experiments on the most performant and widely used WS3 SKU. Experiment Methodology For each experiment we varied the number of pre-warmed instances and maximum burst instances and supplied a steady influx of X messages per minute to the connected Service Bus queue, gradually increase X until the Logic App cannot process all the messages immediately. We push enough (4 million) messages to the queue for each experiment to ensure that each workflow reaches a steady state processing rate. Environment configuration The experiment setup is summarized in the table below: Tests setup Multi Stamp Logic App Number of workflows 1 Templated Triggers Service Bus Trigger batch size 50 Actions Service Bus, Scope, Condition, Compose Number of storage accounts 3 Message size 1 KB Service Bus queue max size 4 GB Service Bus queue message lock duration 5 minutes WS Plan WS3 Service Bus queue message max delivery count 10 Experiment results The experiment results are summarized in the table below: Prewarmed Instances Max Burst Instances Message Throughput 1 20 24000 messages/minute 1 60 65000 messages/minute 5 60 65000 messages/minute 10 60 65000 messages/minute 10 100 85000 messages/minute In all the experiments, the Logic App scaled out to the maximum burst instance allowed at steady state. Editor's Note: The actual business logic can affect the number of machines the app scales out to. The performance might also vary based on the complexity of the workflow logic. Findings Understand the scaling and bottlenecks In the horizontal scaling experiments, when the max burst instances count is 60 or above, we no longer observe "dead-letters" being generated. In these cases, the Service Bus trigger can only fetch messages as fast as the workflow actions can process them. As we can observe in the following figure, all messages are processed immediately after they are fetched. Does the scaling speed affect the workload? As we can see below, a Standard Logic app with a prewarmed instance count of 5 can scale out to its maximum scaling of 60 under 10 minutes. The message fetching and message processing abilities scale out together, preventing the generation of “dead-letters.” Also, from the results in our horizontal scaling experiments, we see that having more prewarmed instances does not affect the steady-state throughput of the workflow. Recommendations With these two findings, we recommend keeping the minimum instance number small for cost-saving, without any impact on your peak performance. If a use case requires a higher throughput, the maximum burst instances setting can be set higher to accommodate that. For production workflows, we still recommend having at least two always-ready instances, as they would reduce any potential downtime from reboots.
