Feature: Support manual redaction

[omar-ahmed42]

Description

Manual Redaction:

• Text Selection-based redaction:

  • 
  • Users can now redact currently selected text by selecting the text then clicking ctrl + s shortcut or by pressing on apply/save/disk icon in the toolbar.
  • Users can delete/cancel the redacted area by clicking on the box containing the text, then clicking on delete/trash icon or by using the shortcut delete.
  • Users can customize the color of the redacted area/text (after the redaction was applied) by simply clicking on the box containing the text/area then clicking on the color palette icon and choosing the color they want.
  • Users can choose to select the color of redaction before redacting text or applying changes (this only affects newly created redaction areas, to change the color of an existing one; check the previous bullet point).

• Draw/Area-based redaction:

  • 
  • Users can now redact an area in the page by selecting the then clicking ctrl + s shortcut or by pressing on apply/save/disk icon in the toolbar.
  • Users can delete/cancel the redacted area by clicking on the drawn box, then clicking on delete/trash icon or by using the shortcut delete (requires temporarily turning off drawing mode).
  • Users can customize the color of the redacted area (after the redaction was applied) by simply clicking on the box containing the area then clicking on the color palette icon and choosing the color they want.
  • Users can choose to select the color of redaction before drawing the box or applying changes (this only affects newly created redaction areas, to change the color of an existing one; check the previous bullet point).

• Page-based redaction:

  • 
  • Users can now redact ENTIRE pages by specifying the page number(s), range(s) or functions.
  • Users can customize the color of page-based redaction (doesn't affect text-based nor draw-based redactions).

Redaction modes:

There are three modes of redaction/operation currently supported

• Text Selection-based redaction (TEXT)
• Draw/Area-based redaction (DRAWING)
• None - by simply not choosing any of the above modes (NONE).

How to use:

• Text Selection-based redaction: click on this icon in the toolbar to enable text-selection redaction mode then select the text you want to redact then press ctrl + s or click on the disk/save icon .

• Draw/Area-based redaction: click on this icon in the toolbar to enable draw/area-based redaction then left mouse click (LMB) on the starting point of the rectangle, then once you are satisfied with the rectangle's placement/dimensions then left mouse click (LMB) again to apply the redaction.

  • Example: Left mouse click (LMB) then move mouse to the right then bottom then Left mouse click (LMB).
  • Note: Red box/rectangle borders indicate that you have not yet saved (you need to left click on the page to save) once saved the borders will become green (they also become clickable/hover-able when drawing mode is off).

• Page-based redactions:: Insert the page number(s), range(s) and/or functions (separated by ,) then select your preferred color and click on Redact to submit.

• Color Customizations:

  • You can change the redaction color for new redactions by clicking on this icon in the toolbar .
  • You can change the redaction color for existing redactions by hovering over the redaction box then clicking on it (Left mouse click LMB) then clicking on color palette (highlighted in red in the picture) then select your preferred color.

• Deletions:

  • You can delete a redacted area by hovering over the redaction box then clicking on it (Left mouse click LMB) then clicking on the trash icon (highlighted in red in the picture) .

Card in the home page:

Closes #465

Checklist

• [x] I have read the Contribution Guidelines
• [x] I have performed a self-review of my own code
• [x] I have attached images of the change if it is UI based
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] If my code has heavily changed functionality I have updated relevant docs on Stirling-PDFs doc repo
• [ ] My changes generate no new warnings
• [ ] I have read the section Add New Translation Tags (for new translation tags only)

[Frooodle]

/deploypr

[github-actions[bot]]

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://185.252.234.121:2433

This deployment will be automatically cleaned up when the PR is closed.

[reecebrowne]

Hi, this feature is absolutely fantastic, great work! I just have questions about presentation. Is there a practical reason you used the pdfjs-legacy pdf viewer instead of using the implementation used on pages like sign? As it is this feature presents totally differently to our other editing tools and may be a bit difficult to use, especially with how small the icons are for the redaction tools. If you could look in to making it look more like the Sign feature or explaining the potential issues if there are technical reasons for doing it the way you have that would be hugely appreciated. Thanks!

[omar-ahmed42]

Hi, before I explain anything let's take a look at what we need first to implement this feature (redaction):

1. To be able to select text.
2. To be able to draw shapes (mainly rectangles).
3. Zooming is required as some text might be really small (because of font size) or because accuracy is required maybe to redact an area of the page (such as parts of an image).

Let's address each point, as for :

1. To be able to select text. If we used the implementation provided in sign page for example, we will run into the problem of not being to select text as the provided view is simply a canvas without any text (if you right click on it, you can store it as an image) so basically it's treated as an image, which would be fail in our use case, we will need to add extra logic and complexity just to display text in the canvas which might prove error-prone, on the other hand, viewer.mjs (used by pdf-js for the viewing pdfs) actually provides the canvas as well as the text on it placed where text is supposed to be placed (in short, canvas with pdf's text). so by using viewer.mjs in this case, we've what we needed ready for us along with a toolbar that we can use to add more tools or hide existing tools.

as for:

2. To be able to draw shapes (mainly rectangles).

This might not be a problem for either, the sign implementation nor the viewer.mjs

as for zooming:

3. Zooming is required as some text might be really small (because of font size) or because accuracy is required maybe to redact an area of the page (such as parts of an image).

We will still have to modify the code and ensure the presentation doesn't lose any quality in the process of zooming in (you can try zooming in the sign - after extracting the canvas and its container out of the form card of course so that it would be bigger - you will find that it loses quality in the process, that's something we have to take into account as well), in other words, we will have to reinvent the wheel to some degree to get the same functionality that's made and ready to use in viewer.mjs

These are the main reasons why I went with viewer.mjs rather than sign's implementation, I believe the sign's implementation is superb and really fits the use case of sign and other pages as well, in the case of this feature I didn't feel like it would be a great fit for this use case.

Another thing I had in mind, is that we might add extra features to the manual redact page, such as displaying on the right the selected text for redaction for example, and on click the page would be displayed or that the user could simply remove it from that list/catalogue on the right side of the screen, also providing faster access to other pages through the ready made page preview on the left of the screen (I am not sure if that's what it's called), we can also modify the page preview to correspond to changes on the actual page (if the user decided to redact some text with a green rectangle, then a green rectangle would appear in the page preview in an approximate location), etc.

We could also modify the look and feel of the toolbar later on to correspond more to the Stirling's look and feel.

As for the icons' size, I can try and making them bigger if you think they would be easier to use, along with the toolbar, either way, I think we would need to come out with better icons for redactions because I don't find the icons that I used to be intuitive enough so that beginners could use it without heading to a guide/tutorial (I couldn't really find any icons to denote the usage of these buttons/features, so I had to come up with a custom solution that can relate even 1% to their intention).

Also, given this new UI for redactions we could later on incorporate auto redactions as well, so that we would've a complete redaction feature that would support all kinds of redactions, while at the same time, we still keep the auto redaction page because it's more lightweight and has specific usage so that users that need it can simply use it directly without the extra complexity of the current page.

Anyway, if you have any feedback, questions or any ideas, please, let me know.

Also, regarding:

this feature is absolutely fantastic, great work!

Thanks!

[reecebrowne]

Hi there. Thanks for your reply, that all makes perfect sense! If you could make those icons bigger that would help straight away and if there's anything else you feel you could do to bring that page a little closer visually to the UI of other pages it would be much appreciated.

I don't know how much can be done about the top toolbar in the pdfjs viewer, would it be possible to pull that functionality out into a custom toolbar, one similar to the one we have in multitool perhaps? If that is possible it would make a world of difference on bringing the presentation more in line with our existing design.

I shall try to find you some icons that will be appropriate for these functions and send them across.

Thanks again for your contributions!

[omar-ahmed42]

If you could make those icons bigger that would help straight away

Sure thing! I think this might provide a better user-experience, so you have a point!

if there's anything else you feel you could do to bring that page a little closer visually to the UI of other pages it would be much appreciated.

Sure, I can try this again, might look better than the current one given the bigger icons, etc

I don't know how much can be done about the top toolbar in the pdfjs viewer, would it be possible to pull that functionality out into a custom toolbar, one similar to the one we have in multitool perhaps? If that is possible it would make a world of difference on bringing the presentation more in line with our existing design.

All I know that we can customize the styling of it, but not pull our the functionality itself into a separate one, not to mention that if we managed to somehow pull it out and make it work:

1. Which area would be most appropriate to the user (in terms of functionality, such as reach, if we place it too far, it would be unpleasant user experience (unless the user uses keyboard shortcuts most of the time), if we place it too close, it might get in the way of the user?
2. Which area would be most pleasant to look at as well for the user (look-wise)?

But as for styling it, I was actually thinking of styling it to match the rest of the designs actually as you mentioned, but I thought I would push the feature first in case we needed to deliver it earlier than expected.

But yeah, I bringing the toolbar and icons as close as possible to existing design should be the next step (after making the icons bigger), so basically I should be starting with the size of the icons then proceed to the toolbar and the tools as well.

I shall try to find you some icons that will be appropriate for these functions and send them across.

Appreciate that a lot! this would be of great help! Thank you!

[reecebrowne]

/deploypr

[github-actions[bot]]

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://185.252.234.121:2433

This deployment will be automatically cleaned up when the PR is closed.

[omar-ahmed42]

Note: Zoom bar and Sidebar were intentionally set to be dark, as the light version of them didn't look good in my opinion.

Also, does the project follow any specific naming conventions in the frontend? (mainly css) if yes, then let me know so that I could rename my classes/ids.

If there's no styling conventions yet in general, then I recommend we start setting them for the frontend and the backend for more consistency in the long term

[reecebrowne]

Hi mate, the new UI looks great thanks for making the changes. As far as the icons go I see you have tried to create custom icons using ink eraser on top of other icons. While I understand why ,I think the base icons without the eraser are fine, we already know the context that this is the redaction page so the icons don't need to be redaction specific but rather tell the use what kind of redaction it is. I think text selection you have already chosen the icon that I would choose

https://fonts.google.com/icons?selected=Material+Symbols+Outlined:text_select_start:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=text+selec&icon.size=24&icon.color=%23e8eaed just remove the custom overlay of the eraser you have added.

As for the area based selection I think this is the slightly better choice of icon https://fonts.google.com/icons?selected=Material+Symbols+Outlined:pageless:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=area&icon.size=24&icon.color=%23e8eaed as it still conveys the box idea but is a bit more obviously an icon than just a square. As with the other icon I don't think the custom eraser version is necessary.

I don't think the save icon is clear for applying text base redaction. I think perhaps this is abetter icon https://fonts.google.com/icons?selected=Material+Symbols+Outlined:approval:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=apply&icon.size=24&icon.color=%23e8eaed or perhaps just a simple check https://fonts.google.com/icons?selected=Material+Symbols+Outlined:check:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=tick&icon.size=24&icon.color=%23e8eaed I think this is better placed directly next to the text selection icon and only shown when text select mode is active to make it clear it is related only to that feature and serves no purpose on box redaction

Could you move the "redact" button that downloads the completed file to your toolbar and use this icon for it instead of the word "redact" please? https://fonts.google.com/icons?selected=Material+Symbols+Outlined:download:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=download&icon.size=24&icon.color=%23e8eaed

[reecebrowne]

A further potential improvement would be to have the "apply redaction" button (currently save Icon) appear next to selected text in the page, similar to the delete/colour selection toolbar you have implemented already, this saves the user scrolling up to the top of the page to apply redactions and also makes functionality a little more clear.

Additionally there is a minor bug with the delete/colour select toolbar. Currently it is quite easy for a redaction box/text selection to appear over the toolbar for nearby redactions making the buttons impossible to click

Great progress so far. It's looking good!

[omar-ahmed42]

we already know the context that this is the redaction page so the icons don't need to be redaction specific but rather tell the use what kind of redaction it is.

Makes perfect sense

just remove the custom overlay of the eraser you have added.

Sure

As for the area based selection I think this is the slightly better choice of icon https://fonts.google.com/icons?selected=Material+Symbols+Outlined:pageless:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=area&icon.size=24&icon.color=%23e8eaed as it still conveys the box idea but is a bit more obviously an icon than just a square. As with the other icon I don't think the custom eraser version is necessary.

I agree, this seems much clearer

I think perhaps this is abetter icon https://fonts.google.com/icons?selected=Material+Symbols+Outlined:approval:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=apply&icon.size=24&icon.color=%23e8eaed or perhaps just a simple check https://fonts.google.com/icons?selected=Material+Symbols+Outlined:check:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=tick&icon.size=24&icon.color=%23e8eaed

I feel like the first icon you mentioned fits our use case perfectly, but I might go with the second one as it would be more intuitive to almost all the users

I think this is better placed directly next to the text selection icon and only shown when text select mode is active to make it clear it is related only to that feature and serves no purpose on box redaction

To make sure I understand, let's visualize them as follows:

1. [text-selection-icon][check-icon_hidden][area-selection-icon]
2. On enabling text-selection mode:
   • [text-selection-icon][check-icon-visible][area-selection-icon] Is that is or did I miss something?

Could you move the "redact" button that downloads the completed file to your toolbar and use this icon for it instead of the word "redact" please? https://fonts.google.com/icons?selected=Material+Symbols+Outlined:download:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=download&icon.size=24&icon.color=%23e8eaed

Sure thing!

[omar-ahmed42]

A further potential improvement would be to have the "apply redaction" button (currently save Icon) appear next to selected text in the page

This is actually a fantastic improvement! I might take a look at it but I think this might take some time but that's okay

Additionally there is a minor bug with the delete/colour select toolbar. Currently it is quite easy for a redaction box/text selection to appear over the toolbar for nearby redactions making the buttons impossible to click

Great observation! I will take a look it

[reecebrowne]

I feel like the first icon you mentioned fits our use case perfectly, but I might go with the second one as it would be more intuitive to almost all the users Yes I think the check is probably the better choice, lets go with that

I think this is better placed directly next to the text selection icon and only shown when text select mode is active to make it clear it is related only to that feature and serves no purpose on box redaction

To make sure I understand, let's visualize them as follows:

1. [text-selection-icon][check-icon_hidden][area-selection-icon]

2. On enabling text-selection mode:

   • [text-selection-icon][check-icon-visible][area-selection-icon] Is that is or did I miss something?

Yes I think that will just help to make it really obvious that button is related to the text selection feature and only activates when in that mode. I think maybe consider trying green for that button to make it clearer it's a "confirm" action. Feel free to go with your gut on what feels clear and looks good on that though as it may look off in practice.

[reecebrowne]

A further potential improvement would be to have the "apply redaction" button (currently save Icon) appear next to selected text in the page

This is actually a fantastic improvement! I might take a look at it but I think this might take some time but that's okay

Yeah I don't think it's necessary to the feature, just a nice UI upgrade, so perhaps that could be a suggestion for additional improvements in a new branch after release if you feel it may be a bigger job.

[omar-ahmed42]

I added the download button as you instructed, and I also added a loading spinner for a better user experience as the processing on the server side might take some time for any reason (especially when the conversion option is checked), so a feedback for the user would be provided as the user might think that the page froze. (This spinner entirely revolves around whether the hidden submission button is disabled or not, there are other ways but they would require applying modifications to downloader.js, and I preferred to not modify downloader.js).

As for the "check" button, I added it and made it not displayed unless text selection mode is activated, but I didn't really add it next to the text selection box, why? I attempted to add it next to the text selection button but this would keep shifting the buttons each time it's pressed which I felt might not be pleasant to the user (there's the option of moving the text selection button to the end, but look wise, it looked better next to the drawing button as it is), so I moved the check button to the right of the color palette and made it green, this way it would be visible to the user and catch their attention even though it's 2 buttons away from the text-selection button.

This way the user knows that the text selection's position will never change, this way, it would be a familiar position/placement for the user making it easier in the long term for the user's muscle memory (as it remembers the exact position with minimum mental calculations), at the same time, if we ever needed to add any temporary buttons related to specific features, the user now would know where to find them. (Especially if we were to add undo/redo buttons)

It looks like this:

As for:

Additionally there is a minor bug with the delete/colour select toolbar. Currently it is quite easy for a redaction box/text selection to appear over the toolbar for nearby redactions making the buttons impossible to click

I successfully fixed

Yeah I don't think it's necessary to the feature, just a nice UI upgrade, so perhaps that could be a suggestion for additional improvements in a new branch after release if you feel it may be a bigger job.

I agree, it's really a nice UI upgrade, I might work on it in a new branch as I feel like this one might take some time. Also, if you have any notes, ideas or questions, please, let me know!

I also wanted to say thanks for your notes! they were helpful!

[reecebrowne]

/deploypr

[github-actions[bot]]

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://185.252.234.121:2433

This deployment will be automatically cleaned up when the PR is closed.

[reecebrowne]

Looking better and better. I think you made the right call on the confirm redaction button.

The main issue now is the double scrolling which is a bit confusing (main page scroll and view window scroll). I think perhaps we should ditch the drag and drop file uploader for this feature and integrate a file upload into the viewer instead, alongside any other features that are still in the window above the viewer. This way the viewer can take up the whole page below the Stirling PDF toolbar, I think that will be a cleaner layout for this feature

[Frooodle]

/deploypr :)

[github-actions[bot]]

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://185.252.234.121:2433

This deployment will be automatically cleaned up when the PR is closed.

[reecebrowne]

Hi @omar-ahmed42 This is looking good. Am I correct in saying that the following is a tool for redacting entire pages?

I think there is some clarity lacking here either way.

Also, could we bring back the Stirling toolbar at the top, there's no way off this page right now.

[omar-ahmed42]

Hi @reecebrowne

This is looking good. Am I correct in saying that the following is a tool for redacting entire pages?

It is, I thought of calling it "Page-based Redaction" but this won't represent the checkbox "Convert PDF to PDF-Image ... etc", so I thought I would go with "Properties" to be more generic.

Also, could we bring back the Stirling toolbar at the top, there's no way off this page right now.

Bringing the nav bar back would bring the second scroll back 😅, we could instead make Stirling clickable and redirect the user to the home page, and upon clicking on it, a popup should come up "You will be redirected to the home page ["Cancel"] ["Proceed"]", why the popup? there's a possibility of mis-clicking on the Stirling icon by the user and if that happens the user will lose all the progress made so far, so we can say that's some kind of a protective measure, what do you think?

[reecebrowne]

It is, I thought of calling it "Page-based Redaction" but this won't represent the checkbox "Convert PDF to PDF-Image ... etc", so I thought I would go with "Properties" to be more generic.

I think just make convert to image a seperate option on the main toolbar rather than in this submenu

Bringing the nav bar back would bring the second scroll back 😅, we could instead make Stirling clickable and redirect the user to the home page, and upon clicking on it, a popup should come up "You will be redirected to the home page ["Cancel"] ["Proceed"]", why the popup? there's a possibility of mis-clicking on the Stirling icon by the user and if that happens the user will lose all the progress made so far, so we can say that's some kind of a protective measure, what do you think?

It shouldn't bring back the scroll issue if handled right, you can turn off scroll on the page and you can set the height of the viewer dynamically to be 100% of parent - the height of the nav to prevent overflow. Here's a mockup I put together with a couple of quick css edits in the browser. No second scroller needed. Can talk you through it if you need.

[omar-ahmed42]

I think just make convert to image a seperate option on the main toolbar rather than in this submenu

I'm thinking more of a dropdown? this way the user would know what this option does (if it's just a button the user might not figure out what it does without a tutorial)

or we can have 2 buttons, 1 for page based redactions (the one above), and the other would be a generic button containing options and fields related to the file or the operations in general, why? We might need in the future to integrate extra features that might not have a specific category into the feature, example: output filename, in this case, the user will find the generic ones over there, what do you think?

Example:

• The circle denotes the button for page-based redaction.
• The square denotes the properties/settings button.

Let me know what you think

It shouldn't bring back the scroll issue if handled right, you can turn off scroll on the page and you can set the height of the viewer dynamically to be 100% of parent - the height of the nav to prevent overflow.
Here's a mockup I put together with a couple of quick css edits in the browser. No second scroller needed. Can talk you through it if you need.

I think I managed to accomplish that after reading your instructions, thanks for your help!

[reecebrowne]

I'm thinking more of a dropdown? this way the user would know what this option does (if it's just a button the user might not figure out what it does without a tutorial)

Convert to image should be a simple enough concept to convey via a tooltip, we also have an existing icon for it so I wouldn't be too worried about this.

or we can have 2 buttons, 1 for page based redactions (the one above), and the other would be a generic button containing options and fields related to the file or the operations in general, why? We might need in the future to integrate extra features that might not have a specific category into the feature, example: output filename, in this case, the user will find the generic ones over there, what do you think?

Page based redactions should definitely be it's own thing, some kind of dropdown or submenu is necessary for functionality but it's not really something I think should be bundled into a settings/properties menu. If you think there's some value in a settings menu for other reasons in additional features I'm not against the menu existing but I think you're bundling key features away in a slightly confusing place with the stuff that's already in. Stuff like output filename is a good example of something that would make sense in such a submenu, key redaction features should be directly in the toolbar

I think I managed to accomplish that after reading your instructions, thanks for your help!

Code looks good, thanks for sorting that.

[omar-ahmed42]

I agree with you everything you mentioned.

Toolbar: You can see that I added a new button as instructed, when it's in green it indicates being "checked" when in red, it indicates being unchecked:

Page based redaction:

[reecebrowne]

/deploypr

[github-actions[bot]]

🚀 PR Test Deployment

Your PR has been deployed for testing!

🔗 Test URL: http://185.252.234.121:2433

This deployment will be automatically cleaned up when the PR is closed.

[reecebrowne]

Looking good!

Thinking the following icon for full page redaction https://fonts.google.com/icons?selected=Material+Symbols+Outlined:document_scanner:FILL@0;wght@400;GRAD@0;opsz@24&icon.query=page&icon.size=24&icon.color=%23e8eaed

Rotated pages aren't working at the moment, see below.

I know we use it elsewhere but the drag box for box selection isn't great, it would be much better if it was click then drag, the current behaviour was weird. There may be some library out there you could use instead. Would be great if you can look into this but it is a bit out of scope so no pressure if you don't fancy it.