allpairspy
allpairspy copied to clipboard
thombashi
Insufficient combination when filter function is specified?
Hello, thank you for providing a great library! I tried to use filter function to cut invalid combination, but it seems generated combinations are insufficient.
# Generate combinations of every supported Python + NumPy version combination.
from allpairspy import AllPairs
numpy = ['1.9', '1.10', '1.11', '1.12', '1.13', '1.14']
python = ['2.7', '3.4', '3.5', '3.6']
def _validate(numpy, python):
valid = []
if numpy == '1.9':
valid = ['2.7', '3.4']
elif numpy == '1.10':
valid = ['2.7', '3.4']
elif numpy == '1.11':
valid = ['2.7', '3.4', '3.5']
elif numpy == '1.12':
valid = ['2.7', '3.4', '3.5', '3.6']
elif numpy == '1.13':
valid = ['2.7', '3.4', '3.5', '3.6']
elif numpy == '1.14':
valid = ['2.7', '3.4', '3.5', '3.6']
return python in valid
def _filter_func(row):
if len(row) == 2:
stat = _validate(*row)
#print('Valid? ', row, ' -> ', stat)
return stat
return True
if __name__ == '__main__':
for i, pairs in enumerate(AllPairs([numpy, python], filter_func=_filter_func)):
print(i, ':', pairs)
I got:
0 : ['1.9', '2.7']
1 : ['1.10', '2.7']
2 : ['1.11', '2.7']
3 : ['1.12', '2.7']
4 : ['1.13', '2.7']
5 : ['1.14', '2.7']
6 : ['1.14', '3.4']
7 : ['1.13', '3.4']
8 : ['1.12', '3.4']
9 : ['1.11', '3.4']
10 : ['1.10', '3.4']
11 : ['1.9', '3.4']
It seems Python 3.5/3.6 variations are totally ignored. Am I missing something? Any suggestions are welcome. Thanks in advance!
Hello, @kmaehashi
AllPairs may not properly work when you try to generate combinations from two parameters.
filter_func will properly work if you use three or more parameters as following (a little bit modified your sample code):
from allpairspy import AllPairs
numpy = ['1.9', '1.10', '1.11', '1.12', '1.13', '1.14']
python = ['2.7', '3.4', '3.5', '3.6']
os = ["Linux", "Windows", "macOS"]
def _validate(row):
numpy, python = row[0], row[1]
valid = []
if numpy == '1.9':
valid = ['2.7', '3.4']
elif numpy == '1.10':
valid = ['2.7', '3.4']
elif numpy == '1.11':
valid = ['2.7', '3.4', '3.5']
elif numpy == '1.12':
valid = ['2.7', '3.4', '3.5', '3.6']
elif numpy == '1.13':
valid = ['2.7', '3.4', '3.5', '3.6']
elif numpy == '1.14':
valid = ['2.7', '3.4', '3.5', '3.6']
return python in valid
def _filter_func(row):
if len(row) > 2:
stat = _validate(row)
#print('Valid? ', row, ' -> ', stat)
return stat
return True
if __name__ == '__main__':
for i, pairs in enumerate(AllPairs([numpy, python, os], filter_func=_filter_func)):
print(i, ':', pairs)
0 : ['1.9', '2.7', 'Linux']
1 : ['1.10', '3.4', 'Linux']
2 : ['1.11', '3.5', 'Linux']
3 : ['1.12', '3.6', 'Linux']
4 : ['1.13', '3.6', 'Windows']
5 : ['1.14', '3.5', 'Windows']
6 : ['1.14', '3.4', 'macOS']
7 : ['1.13', '2.7', 'macOS']
8 : ['1.12', '2.7', 'Windows']
9 : ['1.11', '3.4', 'Windows']
10 : ['1.10', '2.7', 'Windows']
11 : ['1.9', '3.4', 'Windows']
12 : ['1.9', '3.4', 'macOS']
13 : ['1.10', '3.4', 'macOS']
14 : ['1.11', '2.7', 'macOS']
15 : ['1.12', '3.5', 'macOS']
16 : ['1.13', '3.4', 'Linux']
17 : ['1.14', '3.6', 'macOS']
18 : ['1.14', '2.7', 'Linux']
product with filter functions might be more suitable for your sample data:
from allpairspy import AllPairs
from itertools import product
numpy = ['1.9', '1.10', '1.11', '1.12', '1.13', '1.14']
python = ['2.7', '3.4', '3.5', '3.6']
def _validate(numpy, python):
valid = []
if numpy == '1.9':
valid = ['2.7', '3.4']
elif numpy == '1.10':
valid = ['2.7', '3.4']
elif numpy == '1.11':
valid = ['2.7', '3.4', '3.5']
elif numpy == '1.12':
valid = ['2.7', '3.4', '3.5', '3.6']
elif numpy == '1.13':
valid = ['2.7', '3.4', '3.5', '3.6']
elif numpy == '1.14':
valid = ['2.7', '3.4', '3.5', '3.6']
return python in valid
def _filter_func(row):
return _validate(*row)
if __name__ == '__main__':
for i, pairs in enumerate(filter(_filter_func, product(numpy, python))):
print(i, ':', pairs)
0 : ('1.9', '2.7')
1 : ('1.9', '3.4')
2 : ('1.10', '2.7')
3 : ('1.10', '3.4')
4 : ('1.11', '2.7')
5 : ('1.11', '3.4')
6 : ('1.11', '3.5')
7 : ('1.12', '2.7')
8 : ('1.12', '3.4')
9 : ('1.12', '3.5')
10 : ('1.12', '3.6')
11 : ('1.13', '2.7')
12 : ('1.13', '3.4')
13 : ('1.13', '3.5')
14 : ('1.13', '3.6')
15 : ('1.14', '2.7')
16 : ('1.14', '3.4')
17 : ('1.14', '3.5')
18 : ('1.14', '3.6')
@thombashi Thank you for your help. According to the output from your code, when focusing on Python = 3.5 combination,
2 : ['1.11', '3.5', 'Linux']
5 : ['1.14', '3.5', 'Windows']
15 : ['1.12', '3.5', 'macOS']
NumPy = 1.13 & Python = 3.5 combination, which is not expected to be filtered, seems missing. Is this an expected behavior?
@kmaehashi Thank you for your comment. That is unexpected behavior. I will check into it.
@thombashi Thanks for the quick response! Actually I was trying to generate combinations of 8 axes with filtering (https://github.com/chainer/chainer-test/pull/403) and found some pairs missing from generated patterns. (The first example (NumPy/Python) I wrote was the minimum code to reproduce the issue, not the actual usecase.)
@kmaehashi Thank you for additional information. This issue might take some time (I had little time to solve the issue for now).
@thombashi Also, the example generation provided in the README lacks following pairs:
98 - Hourly
98 - Contr.
98 - 10
98 - 30
98 - 60
NT - 15
2000 - 6
XP - 10
Salaried - 30
Hourly - 30
Hourly - 60
Part-Time - 60
Are there any plans to fix it?
Fixed in this PR.
For @kmaehashi's original input the result is:
{numpy=1.9, python=2.7}
{numpy=1.1, python=2.7}
{numpy=1.11, python=2.7}
{numpy=1.12, python=2.7}
{numpy=1.13, python=2.7}
{numpy=1.14, python=2.7}
{numpy=1.14, python=3.4}
{numpy=1.13, python=3.4}
{numpy=1.12, python=3.4}
{numpy=1.11, python=3.4}
{numpy=1.1, python=3.4}
{numpy=1.9, python=3.4}
{numpy=1.11, python=3.5}
{numpy=1.12, python=3.5}
{numpy=1.13, python=3.5}
{numpy=1.14, python=3.5}
{numpy=1.14, python=3.6}
{numpy=1.13, python=3.6}
{numpy=1.12, python=3.6}
Code:
parameters = OrderedDict(
{"x1": [True, False],
"x2": [True, False]})
def constraints(row):
n = len(row)
if n > 1:
if row[0] is False and row[1] is False:
return False
return True
print("PAIRWISE:")
res = AllPairs(parameters, filter_func=constraints, n=2)
Desired Output:
0 Pairs(x1=True, x2=True)
1 Pairs(x1=False, x2=True)
2 Pairs(x1=True, x2=False)
Actual Output:
0 Pairs(x1=True, x2=True)
1 Pairs(x1=False, x2=True)
