binary.com-interview-question icon indicating copy to clipboard operation
binary.com-interview-question copied to clipboard

Support high-frequency-trading

Open englianhu opened this issue 3 years ago • 1 comments

Screenshot_1


Best RAM You Can Buy Today Patriot Viper Steel DDR4-4400 (2x 8GB) ... Patriot Viper RGB DDR4-3600 (2x 8GB) ... Patriot Viper 4 DDR4-3400 (2x 8GB) ... Corsair Vengeance RGB Pro DDR4-3200 (4x 8GB) ... Patriot Viper Steel DDR4-3200 (2x 16GB) ... Patriot Viper Steel DDR4-3600 C18 (2x 32GB) ... G. ... Corsair Vengeance LPX DDR4-2666 (2x 8GB) Oct 20, 2020 Best RAM 2020: Fast, Cheap and RGB | Tom's Hardware


Nvidia GeForce RTX 3080. The best graphics card for PC gaming right now. ... Nvidia GeForce RTX 2070 Super. The best 4K graphics card for reasonable money. ... AMD Radeon RX 5700. The best 1440p graphics card… with a little work. ... AMD Radeon RX 5600 XT. ... Nvidia GeForce GTX 1660 Super. ... Nvidia GeForce GTX 1650 Super. Oct 21, 2020 The best graphics cards in 2020 | PC Gamer

Need to upgraded my equipement to enchance the efficiency...

Originally posted by @englianhu in https://github.com/englianhu/report/issues/9#issuecomment-719737291

englianhu avatar Nov 26 '20 08:11 englianhu

https://github.com/JinmiaoChenLab/cytofkit/issues/17#issuecomment-416047706 and https://github.com/JinmiaoChenLab/cytofkit/issues/17#issuecomment-608450415

> mts <- smp %>% 
+     msts(seasonal.periods = c(1440, nrow(smp)))
>   
>   mts <- llply(1:ncol(mts), function(i) {
+     y <- mts[,i] %>% 
+       tbats %>% 
+       forecast(h = 1440) %>% 
+       as_tibble
+       names(y)[1] <- names(smp)[i]
+       y
+     }) %>% 
+     bind_rows %>% 
+     mutate(Model = factor('tbats'), Period = factor('dy.qt'), type = case_when(
+       !is.na(open) ~ 'open', 
+       !is.na(high) ~ 'high', 
+       !is.na(low) ~ 'low', 
+       !is.na(close) ~ 'close')) %>% 
+     dlply(.(type, Period), function(x) {
+       x %<>% dplyr::rename(open.Point.Forecast = open, 
+                            high.Point.Forecast = high, 
+                            low.Point.Forecast = low, 
+                            close.Point.Forecast = close)
+       names(x)[str_detect(names(x), '80|95')] <- paste0(x$type[1], '.', names(x)[str_detect(names(x), '80|95')])
+       x[colSums(!is.na(x)) > 0] %>% 
+         data.frame %>% 
+         as_tibble %>% 
+         dplyr::select(-type)
+       
+       }) %>% 
+     join_all %>% 
+     as_tibble
Joining by: Model, Period
Joining by: Model, Period
- Error: cannot allocate vector of size 11.1 Gb
>   memory.size()
[1] 11984.01
> gc()
           used  (Mb) gc trigger    (Mb)   max used    (Mb)
Ncells  3767976 201.3    7315067   390.7    7315067   390.7
Vcells 12814470  97.8 1474157596 11247.0 1534873241 11710.2
> memory.limit()
[1] 16274

memory.size() occupied 75% of memory.limit()

> mts <- smp %>% 
+     msts(seasonal.periods = c(1440, nrow(smp)))
>   
>   mts <- llply(1:ncol(mts), function(i) {
+     y <- mts[,i] %>% 
+       tbats %>% 
+       forecast(h = 1440) %>% 
+       as_tibble
+       names(y)[1] <- names(smp)[i]
+       y
+     }) %>% 
+     bind_rows %>% 
+     mutate(Model = factor('tbats'), Period = factor('dy.qt'), type = case_when(
+       !is.na(open) ~ 'open', 
+       !is.na(high) ~ 'high', 
+       !is.na(low) ~ 'low', 
+       !is.na(close) ~ 'close')) %>% 
+     dlply(.(type, Period), function(x) {
+       x %<>% dplyr::rename(open.Point.Forecast = open, 
+                            high.Point.Forecast = high, 
+                            low.Point.Forecast = low, 
+                            close.Point.Forecast = close)
+       names(x)[str_detect(names(x), '80|95')] <- paste0(x$type[1], '.', names(x)[str_detect(names(x), '80|95')])
+       x[colSums(!is.na(x)) > 0] %>% 
+         data.frame %>% 
+         as_tibble %>% 
+         dplyr::select(-type)
+       
+       })
> mtss <- mts %>% bind_cols
New names:
* Model -> Model...6
* Period -> Period...7
* Model -> Model...13
* Period -> Period...14
* Model -> Model...20
* ...
> mtss
# A tibble: 1,440 x 28
   close.Lo.80 close.Hi.80 close.Lo.95 close.Hi.95 close.Point.For~ Model...6
         <dbl>       <dbl>       <dbl>       <dbl>            <dbl> <fct>    
 1        118.        118.        118.        118.             118. tbats    
 2        118.        118.        118.        118.             118. tbats    
 3        118.        118.        118.        118.             118. tbats    
 4        118.        118.        118.        118.             118. tbats    
 5        118.        118.        118.        118.             118. tbats    
 6        118.        118.        118.        118.             118. tbats    
 7        118.        118.        118.        118.             118. tbats    
 8        118.        118.        118.        118.             118. tbats    
 9        118.        118.        118.        118.             118. tbats    
10        118.        118.        118.        118.             118. tbats    
# ... with 1,430 more rows, and 22 more variables: Period...7 <fct>,
#   high.Lo.80 <dbl>, high.Hi.80 <dbl>, high.Lo.95 <dbl>, high.Hi.95 <dbl>,
#   high.Point.Forecast <dbl>, Model...13 <fct>, Period...14 <fct>,
#   low.Lo.80 <dbl>, low.Hi.80 <dbl>, low.Lo.95 <dbl>, low.Hi.95 <dbl>,
#   low.Point.Forecast <dbl>, Model...20 <fct>, Period...21 <fct>,
#   open.Point.Forecast <dbl>, open.Lo.80 <dbl>, open.Hi.80 <dbl>,
#   open.Lo.95 <dbl>, open.Hi.95 <dbl>, Model...27 <fct>, Period...28 <fct>
> mtss[str_detect(names(mtss), 'Model.|Period.')]
# A tibble: 1,440 x 8
   Model...6 Period...7 Model...13 Period...14 Model...20 Period...21 Model...27
   <fct>     <fct>      <fct>      <fct>       <fct>      <fct>       <fct>     
 1 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 2 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 3 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 4 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 5 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 6 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 7 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 8 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
 9 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
10 tbats     dy.qt      tbats      dy.qt       tbats      dy.qt       tbats     
# ... with 1,430 more rows, and 1 more variable: Period...28 <fct>
> mtss[str_detect(names(mtss), 'Model.|Period.')] <- NULL
> mtss

I used bind_cols() as alternative method to solve my issue, there has different methods depends on your issue (example: normally matrix will be more efficient than data.frame)...

lineprof in Advanced R by Hadley Wickham microbenchmark in Advanced R by Hadley Wickham

There has eventually need to use better pc for high efficiency computing...

englianhu avatar Nov 26 '20 09:11 englianhu