Remove check over method name

[Prasannajnaeyulu]

• The method name from which expect being called not necessiarly be test*; Not all frameworks warrant the method names starts with test*

[pr4bh4sh]

But how do we come to the conclusion that the caller method should always start with test*

I use unittest or pytest which uses 'test' in the test method name to classify a method as a test method. The library is not tested with other testing frameworks as wasn't a requirement for me and was reported as an issue until you raised it.

I would say, Instead of throwing the exception which is a breaking change, why don't we set the caller to what it was previously incase you couldn't find any method that starts with test* in the stack. Let me change this MR inline to what I am saying.

I'm not in favour of this. If we don't know which test method the stack trace belongs to it will create more confusion for novice users. They will not be able to figure out the root cause as it will mess the result. In the current implementation, the exception is raised to prevent that situation.

What I propose is to identify the runner method for the major python testing framework and then extract the actual test method based on that. e.g. the method call chain for pytest

caller = {list: 44} ['expect', 'verify', 'test_MethodChainCall', 'inner', 'pytest_pyfunc_call', '_multicall', '<lambda>', '_hookexec', '__call__', 'runtest', 'pytest_runtest_call', '_multicall', '<lambda>', '_hookexec', '__call__', '<lambda>', 'from_call', 'call_runtest_hook'
 00 = {str} 'expect'
 01 = {str} 'verify'
 02 = {str} 'test_MethodChainCall'
 03 = {str} 'inner'
 04 = {str} 'pytest_pyfunc_call'
 05 = {str} '_multicall'
 06 = {str} '<lambda>'
 07 = {str} '_hookexec'
 08 = {str} '__call__'
 09 = {str} 'runtest'
 10 = {str} 'pytest_runtest_call'
 11 = {str} '_multicall'
 12 = {str} '<lambda>'
 13 = {str} '_hookexec'
 14 = {str} '__call__'
 15 = {str} '<lambda>'
 16 = {str} 'from_call'
 17 = {str} 'call_runtest_hook'
 18 = {str} 'call_and_report'
 19 = {str} 'runtestprotocol'
 20 = {str} 'pytest_runtest_protocol'
 21 = {str} '_multicall'
 22 = {str} '<lambda>'
 23 = {str} '_hookexec'
 24 = {str} '__call__'
 25 = {str} 'pytest_runtestloop'
 26 = {str} '_multicall'
 27 = {str} '<lambda>'
 28 = {str} '_hookexec'
 29 = {str} '__call__'
 30 = {str} '_main'
 31 = {str} 'wrap_session'
 32 = {str} 'pytest_cmdline_main'
 33 = {str} '_multicall'
 34 = {str} '<lambda>'
 35 = {str} '_hookexec'
 36 = {str} '__call__'
 37 = {str} 'main'
 38 = {str} '<module>'
 39 = {str} 'execfile'
 40 = {str} '_exec'
 41 = {str} 'run'
 42 = {str} 'main'
 43 = {str} '<module>'
 __len__ = {int} 44

and for unittest

caller = {list: 21} ['expect', 'verify', 'testMethodChainCall', 'inner', '_callTestMethod', 'run', '__call__', 'run', '__call__', 'run', '__call__', 'run', 'run', 'runTests', '__init__', '<module>', 'execfile', '_exec', 'run', 'main', '<module>']
 00 = {str} 'expect'
 01 = {str} 'verify'
 02 = {str} 'testMethodChainCall'
 03 = {str} 'inner'
 04 = {str} '_callTestMethod'
 05 = {str} 'run'
 06 = {str} '__call__'
 07 = {str} 'run'
 08 = {str} '__call__'
 09 = {str} 'run'
 10 = {str} '__call__'
 11 = {str} 'run'
 12 = {str} 'run'
 13 = {str} 'runTests'
 14 = {str} '__init__'
 15 = {str} '<module>'
 16 = {str} 'execfile'
 17 = {str} '_exec'
 18 = {str} 'run'
 19 = {str} 'main'
 20 = {str} '<module>'
 __len__ = {int} 21

You can see in the above call stack that inner method is common method and previous method in the call stack is the actual test method name.

[pr4bh4sh]

These are the possible case for example:

1. stack_list[0].function returns expect. As it's a common method for all result will not be useful.
2. stack_list[1].function will return verify for the last test test_MethodChainCall. The method verify can be shared with multiple test method and this may again render the result useless.
3. stack_list[2].function returns the test method name correctly but we don't know for sure that stack_list[2] in the stack is the correct test method for all tests. As it can be either stack_list[1] or stack_list[2] on the call stack.

[Prasannajnaeyulu]

These are the possible case for example:

1. stack_list[0].function returns expect. As it's a common method for all result will not be useful.
2. stack_list[1].function will return verify for the last test test_MethodChainCall. The method verify can be shared with multiple test method and this may again render the result useless.
3. stack_list[2].function returns the test method name correctly but we don't know for sure that stack_list[2] in the stack is the correct test method for all tests. As it can be either stack_list[1] or stack_list[2] on the call stack.

[Prasannajnaeyulu]

I think the problem you are solving here is to print the very last function call in the stack from which the expect() is called. If so, Why don't we print the whole stack trace? I think user can able to figure out from this full stack what is his method name and where expect() call is routed from. What's the problem with that approach?

[pr4bh4sh]

Sorry for the delayed response, hammered with office work.

The problem is it will mess the stack trace. I will not get much time to test the approch you are suggesting. If you could try it out with a couple of sample tests and post the stack trace here, it would be helpful.

[Prasannajnaeyulu]

Sorry for the delayed response, hammered with office work.

The problem is it will mess the stack trace. I will not get much time to test the approch you are suggesting. If you could try it out with a couple of sample tests and post the stack trace here, it would be helpful.

Sure I’ll give a try with behave framework I am using to see the call stack.