whisper.cpp icon indicating copy to clipboard operation
whisper.cpp copied to clipboard

[feature suggestion] Windows optimizations (segmented heap)

Open eladkarako opened this issue 1 year ago • 1 comments

I've locally patched up recent binaries with this manifest

<?xml version="1.0" encoding="UTF-8" standalone="yes"?> 
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> 
  <application xmlns="urn:schemas-microsoft-com:asm.v3"> 
     <windowsSettings> <longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>                   </windowsSettings> 
     <windowsSettings> <heapType      xmlns="http://schemas.microsoft.com/SMI/2020/WindowsSettings">SegmentHeap</heapType>                 </windowsSettings> 
  </application> 
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"> 
    <security> 
      <requestedPrivileges> 
        <requestedExecutionLevel level="asInvoker" uiAccess="false" /> 
      </requestedPrivileges> 
    </security> 
  </trustInfo> 
  <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1"> 
    <application> 
      <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" /> 
      <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}" /> 
      <supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}" /> 
      <supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}" /> 
      <supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}" /> 
    </application> 
  </compatibility> 
</assembly> 

to make them use Windows 10/11+ new segmented heap
(compatibility is for running them outside the vista virtualization sandbox).

I think it made RAM allocation a lot faster,
it is noticeable especially in the large dataset.

more testing is needed..
I've noticed --verbose is no longer present,
.. any idea for benchmarking tests?

eladkarako avatar Nov 27 '22 11:11 eladkarako

The --verbose flag didn't do anything so I removed it.

For benchmarking, you can use the bench tool #89

ggerganov avatar Nov 27 '22 18:11 ggerganov