Issues while validating locale specific characters.

[meg23]

From [email protected] on September 30, 2010 00:41:47

What steps will reproduce the problem? 1. I have locale specific site with russia,hungary, etc.. 2. there will be special character with locale specific. 3. for english locale its working fine, but locale specific instance it creates problem while validating. it is not accepting special characters. What is the expected output? What do you see instead? It should accept locale specific characters like russian and hungary.... What version of the product are you using? On what operating system? ESAPI-2.0.jar + windows XP Please provide any additional information below. Please let me know what we can do for the same.

Original issue: http://code.google.com/p/owasp-esapi-java/issues/detail?id=156

[meg23]

From [email protected] on September 29, 2010 23:10:02

My point here is that, during internationalization how it will work.

[meg23]

From [email protected] on September 30, 2010 01:15:24

I am getting below error while entering input value for russia locale. Input value = Покупайте подгузники, собирайте баллы и получайте гарантированные подарки

13:43:34,670 INFO ExampleApplication:ESAPIFilter:294 - [Anonymous:null@unknown -> 127.0.0.1:8080/ExampleApplication/ESAPIFilter] GET http://localhost:8080/test/test.jsp 13:43:40,389 INFO ExampleApplication:ESAPIFilter:294 - [Anonymous:312555@unknown -> 127.0.0.1:8080/ExampleApplication/ESAPIFilter] POST http://localhost:8080/test/test1.jsp?lastname=&Submit=Submit Query&firstname=Покупайте подгузники, собирайте баллы и получайте гарантированные подарки 13:43:40,420 WARN ExampleApplication:IntrusionDetector:294 - [Anonymous:312555@unknown -> 127.0.0.1:8080/ExampleApplication/IntrusionDetector] Invalid input: context=HTTP request parameter: firstname, type(HTTPParameterValue)=^[a-zA-Z0-9.!:;@#$%^&{}_+-=,.~' ]*$, input=Покупайте подгузники, собирайте баллы и получайте гарантированные подарки org.owasp.esapi.errors.ValidationException: HTTP request parameter: firstname: Invalid input. Please conform to regex ^[a-zA-Z0-9.!:;@#$%^&{}[]()_+-=,.~' ]*$ with a maximum length of 65535 at org.owasp.esapi.reference.validation.StringValidationRule.getValid(StringValidationRule.java:121) at org.owasp.esapi.reference.DefaultValidator.getValidInput(DefaultValidator.java:172) at org.owasp.esapi.reference.DefaultValidator.assertIsValidHTTPRequest(DefaultValidator.java:692) at org.owasp.esapi.reference.DefaultValidator.isValidHTTPRequest(DefaultValidator.java:662) at com.org.esapi.ESAPIFilterJava.doFilter(ESAPIFilterJava.java:84) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)

[meg23]

From [email protected] on September 30, 2010 01:33:42

My current value for Validator.HTTPParameterValue is as below

Validator.HTTPParameterValue=^[a-zA-Z0-9.-/:;,.?#&+=@_ ]*$

[meg23]

From chrisisbeef on November 01, 2010 09:07:00

The Validators are not configured out-of-the-box to support internationalization. In order to support this you will need to customise the regexp patterns to allow for "valid" characters in languages that you are supporting.

In the future it may be worthwhile to create a I18N compatible validation framework as a plugin for validators, however - this would require analysts with knowledge of what chars are considered valid across any language we support with the plugin, or conversely the plugin would use configuration to allow end-users to configure valid character sets.

IMHO - this is beyond the scope of the 2.0 release and the 2.1 release as well, so I am slotting this for 2.2 for the time being

Status: Accepted
Labels: -Type-Defect Type-Enhancement Milestone-Release2.2 Usability

[meg23]

From [email protected] on November 01, 2010 22:42:13

Hi chrisisbeef,

Thanks for your reply. However we would like to go with below locales for validating characters.

1. Japan
2. Russia
3. China
4. Arabic
5. Hungary
6. French
7. Dutch
8. Finnish
9. Swedish and other middle east country specific languages also.

[kwwall]

Changing this to a 3.0 milestone as it is unlikely that the current team will get to in for the 2.3 release.