scriptoriumadmin/featurebase
1
0
Fork 0
mirror of https://github.com/featurebasedb/featurebase.git synced 2026-10-09 04:17:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Change time estimation to use avg time per message

Browse source 
In [SUP-75](https://molecula.atlassian.net/browse/SUP-75?atlOrigin=eyJpIjoiYmU5MzdkMmUyZTAyNGQ2Y2IzMDMzYTgzMDU2Y2ZhNmMiLCJwIjoiaiJ9) Allen
pointed out that the time estimation is really good for the first couple lines of output, but gets exponentially worse as execution continues.

After looking into it, it looks like we’re currently using a heuristic based on the amount of messages processed in the previous
second(ish) which is what results in that sort of exponential drop off.

To remedy this, I adjusted the time estimation calculation to use the average time per message up to the point of calculating the new
estimate to ideally improve estimates over time, with the trade-off of a potentially less accurate estimate to begin with.
This commit is contained in:
reesporte 2021-10-21 13:09:24 -05:00
parent 3605449b53
commit adf3e528f5
1 changed files with 4 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
server.go
View file

@ -722,8 +722,10 @@ func (s *Server) Open() error {
if now := time.Now(); now.Sub(prevMsg) > time.Second {
progressRatio := float64(i+1) / float64(len(toSend))
remainingRatio := 1 - progressRatio
timeRemaining := time.Duration(float64(now.Sub(prevMsg)) * (remainingRatio / progressRatio))
numSentMessages := len(toSend) - (i + 1)
messagesLeft := len(toSend) - numSentMessages
avgTimePerMessage := float64(now.Sub(start)) / float64(numSentMessages)
timeRemaining := time.Duration(avgTimePerMessage * float64(messagesLeft))
s.logger.Printf("synced %d/%d messages (%.2f%% complete; %s remaining)", i+1, len(toSend), 100*progressRatio, timeRemaining)
prevMsg = now
}