Top
PowerApps

Sketch-noting in PowerApps and merging images


Post was inspired by a comment with a question posted some time ago under the post about hand written signatures. Kevan asked, if this is possible to take a photo, then sketch-note on it and finally save two images as a single one.

I thought that it should be fairly easy and to be honest it was, however I failed on the last step – merging the images. I must really admit it – I didn’t make it on my own. Instead I decided to use free API called “Convert API“. I wanted to make it using Azure Function, but I wasn’t able to do it alone and help didn’t come from anywhere too 🙂

The solution

It is build from the following components:

  1. PowerApps – is used for taking a photo and then overlaying notes on it;
  2. Blob Storage – used for uploading image from a camera and second one – with sketch-notes, as well as for uploading the final, merged image;
  3. Flow – receives all the information about transformations done to the images and overlays and creates HTML merging two images in a single canvas;
  4. Convert API – used to perfectly convert HTML into a PNG file 🙂

Sketch-noting in PowerApps

The idea is simple – first user is taking a picture:

Camera in PowerApps for taking pictures
Camera in PowerApps for taking pictures

Then, can manipulate it (rotate, scale, position) using the available controls:

Image manipulation in PowerApps
Image manipulation in PowerApps

And finally can append notes:

Appending notes to an image
Appending notes to an image

Finally clicks the button to merge two pictures and gets the result:

Confirmation message after merging photo with notes in Microsoft Flow
Confirmation message after merging photo with notes in Microsoft Flow

And a generated file:

Merged image with notes
Merged image with notes

Merging images in PowerApps (and Flow 😉

Flow plays first chair in the solution. First it receives all the information about the transformation done to the image:

Receiving image transformation properties in Flow
Receiving image transformation properties in Flow

Data is passed from PowerApps via function parameters:

Set(
    FileURL,
    Mergedrawingwithphoto.Run(
        "/files/takenNotes.png",
        "/files/takenPhoto.png",
        TakenPhoto.Width,
        TakenPhoto.Height,
        Notes.Width,
        Notes.Height,
        Scale_slider.Value,
        Value(Rotation),
        TakenPhoto.X,
        TakenPhoto.Y
    )
);

Next Flow is downloading photo and notes from the Blob Storage:

Downloading blobs from Blob Storage in Microsoft Flow
Downloading blobs from Blob Storage in Microsoft Flow

Next Flow checks, whether the image was rotated to a vertical position, or is it in horizontal. Based on that it then generates different HTML:

Flow generating HTML with overlaying images
Flow generating HTML with overlaying images

Then it is sending the HTML code to the Convert API service to get back a JSON with PNG file contents:

Passing HTML to ConvertAPI
Passing HTML to Convert API

Lastly, it is parsing the JSON it receives from Convert API and uploads the PNG file contents to Blob Storage, and is returning uploaded file URL back to PowerApps:

Parsing JSON and uploading file to Blob Storage
Parsing JSON and uploading file to Blob Storage
body('Parse_Conversion_Body')?['Files'][0]['FileName']
body('Parse_Conversion_Body')?['Files'][0]['FileData']

The working solution

Check how it works:

PowerApps desktop - merging image with notes
PowerApps desktop – merging image with notes

It does also work fine when using a mobile version:

PowerApps mobile - merging image with notes
PowerApps mobile – merging image with notes

You can find the app published to PowerApps Community Gallery here:
https://powerusers.microsoft.com/t5/Community-Apps-Gallery/App-for-taking-photos-and-adding-notes/m-p/289368#M186


Tomasz Poszytek

Hi, I am Tomasz. I am expert in the field of process automation and business solutions' building using Power Platform. I am Microsoft MVP and Nintex vTE.

8 Comments
  • John Harper

    for disconnected mode can this blob be saved on the mobile until the user sync’s with the 365 network?

    October 30, 2019 at 6:12 pm Reply
    • Tomasz Poszytek

      Hey! That’s a very good question. So basically the signature is being stored in a temporary, local blob storage. You could get it from there and then save in a collection, using the saveData expression (https://docs.microsoft.com/en-us/powerapps/maker/canvas-apps/functions/function-savedata-loaddata). So later, when user opens the app, you can get that signature contents using the loadData expression however I have no idea, if you will be able to load the signature back to the control. For sure you will be able to send it to Flow for example.

      October 30, 2019 at 10:43 pm Reply
  • Jordan Marsh

    Hi Tomasz,
    Thanks for this guide – you have no idea how helpful it’s been to me!
    I’m having an issue with my Flow once I reach the HTTP request portion and was hoping you might have some input. Essentially I’m getting the below error when I send the request to the API:

    “{“Code”:4002,”Message”:”Bad JSon format. The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters. “}”

    I’ve tested sending the Photo and Drawing individually to the API through the HTTP action, and both tests have been successful, so as I see it there’s no issue with the base64 makeup of the images themselves. It just fails when I put both images together inside the ImageHTML variable and try and send that to the API, but as far as I can see I’ve written it exactly the same as how you have in the above guide – the only thing I’m thinking it could be is the expressions? If you don’t mind, would you be able to tell me what your complete expressions were for the DataURI and Add formula’s?

    Any help would be appreciated!
    Thank you

    December 18, 2019 at 6:00 pm Reply
    • Tomasz Poszytek

      Sure! Both dataURI expressions are the same: dataUri(body(‘Photo’)) and dataUri(body(‘Drawing’)) – so the contents of the images from Blob Storage, which names were passed from Power App. Add for the vertical case is: add(variables(‘var_Y_Nmb’), variables(‘var_HeaderHeight_Nmb’)) and for the horizontal: add(variables(‘var_Y_Nmb’), variables(‘var_HeaderHeight_Nmb’)). This is are contents of the horizontal imageHTML variable:

      December 20, 2019 at 8:30 am Reply
  • Markanday Mishra

    Hi Tomasz,

    I imported this app and tried to test but I am getting two issue.

    1. camera is not working once opening this app in the laptop.
    2. The flow is getting failed with error.

    Specified container files does not exist.
    clientRequestId: 4543883b-2af6-40c1-88d6-b459b2ddc742

    I have added the blob azure storage account in the app but still getting the above.

    do we have to change anything in the app or flow?

    June 4, 2020 at 2:36 pm Reply
    • Tomasz Poszytek

      The reason why camera might not be working is that you didn’t give the browser permissions to use it. About the second issue – double check the path you are trying to use to get to the asset. If it is opening in the browser, flow must have an access to it as well.

      June 4, 2020 at 11:34 pm Reply
  • Nicolas Paredes

    Hi Tomaz, great app you’ve bult. Just a question, is possible to do it wihtout using azure blob storage?

    Thanks!

    July 7, 2020 at 5:47 pm Reply

Post a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.