trafodion icon indicating copy to clipboard operation
trafodion copied to clipboard

Published 20 hours ago • verified publisher icon apache

[TRAFODION-3183] fetch huge data give rise to core

Open mashengchen opened this issue 6 years ago • 21 comments

each time do fetch , calc whether fetch size is bigger than 1GB, if true ,devide fetch times depend on fetchsize/1gb

mashengchen avatar Aug 14 '18 03:08 mashengchen

Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/2942/

Traf-Jenkins avatar Aug 14 '18 03:08 Traf-Jenkins

Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/2942/

Traf-Jenkins avatar Aug 14 '18 06:08 Traf-Jenkins

is there some one can review this?

mashengchen avatar Dec 17 '18 05:12 mashengchen

jenkins, retest

DaveBirdsall avatar Dec 20 '18 02:12 DaveBirdsall

@selvaganesang, does this look good to you now?

DaveBirdsall avatar Dec 20 '18 02:12 DaveBirdsall

Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3071/

Traf-Jenkins avatar Dec 20 '18 02:12 Traf-Jenkins

Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/3071/

Traf-Jenkins avatar Dec 20 '18 04:12 Traf-Jenkins

New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3076/

Traf-Jenkins avatar Dec 24 '18 04:12 Traf-Jenkins

Test Failed. https://jenkins.esgyn.com/job/Check-PR-master/3076/

Traf-Jenkins avatar Dec 24 '18 07:12 Traf-Jenkins

New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3077/

Traf-Jenkins avatar Dec 26 '18 11:12 Traf-Jenkins

Test Failed. https://jenkins.esgyn.com/job/Check-PR-master/3077/

Traf-Jenkins avatar Dec 26 '18 13:12 Traf-Jenkins

New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3078/

Traf-Jenkins avatar Dec 27 '18 02:12 Traf-Jenkins

Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/3078/

Traf-Jenkins avatar Dec 27 '18 05:12 Traf-Jenkins

@selvaganesang do you think it's OK now

mashengchen avatar Jan 01 '19 13:01 mashengchen

is there someone can review this?

mashengchen avatar Jan 03 '19 10:01 mashengchen

I think @selvaganesang is still out on vacation; he should be back tomorrow. I will ping him then.

DaveBirdsall avatar Jan 07 '19 18:01 DaveBirdsall

New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3079/

Traf-Jenkins avatar Jan 08 '19 02:01 Traf-Jenkins

Test Failed. https://jenkins.esgyn.com/job/Check-PR-master/3079/

Traf-Jenkins avatar Jan 08 '19 05:01 Traf-Jenkins

Previous Test Aborted. New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/3080/

Traf-Jenkins avatar Jan 08 '19 05:01 Traf-Jenkins

Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/3080/

Traf-Jenkins avatar Jan 08 '19 08:01 Traf-Jenkins

Looks ok with some subtle corrections needed. getFetchSize needs to call setFetchSizeIfExceedLimit() so that it can return correct value of fetch size when getFetchSize is issued without calling next(). You can do some optimization so that setFetchSizeIfExceededLimit is called only once instead of calculating the fetch size every time next() is called. FetchSize can be obtained via a data source property to control the size of the buffer to ship from server to client. Currently it is either 100 rows or 1 GB

selvaganesang avatar Jan 09 '19 18:01 selvaganesang