Skip to content

adding rate limiting to public client subscriptions, increasing subsc… #231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Daniel1984 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

adding rate limiting to public client subscriptions, increasing subsc… #231

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
3.0.4
- Features
- adding subscriptions rate limiting

3.0.3
- Features
- new websocket manager. Please refer to `/examples/ws` for usage examples.
Expand Down
11 changes: 10 additions & 1 deletion pkg/mux/mux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,18 @@ import (
"fmt"
"log"
"sync"
"time"

"github.com/bitfinexcom/bitfinex-api-go/pkg/models/event"
"github.com/bitfinexcom/bitfinex-api-go/pkg/mux/client"
"github.com/bitfinexcom/bitfinex-api-go/pkg/mux/msg"
)

const (
rateLimit = 60 * 1000 / 20 // 20 connections per minute
pubClientSubsLimit = 30
)

// Mux will manage all connections and subscriptions. Will check if subscriptions
// limit is reached and spawn new connection when that happens. It will also listen
// to all incomming client messages and reconnect client with all its subscriptions
Expand Down Expand Up @@ -105,6 +111,9 @@ func (m *Mux) Subscribe(sub event.Subscribe) *Mux {
m.mtx.Lock()
defer m.mtx.Unlock()

// make sure we don't hit the rate limit
time.Sleep(rateLimit * time.Millisecond)
Copy link
Contributor

@robertkowalski robertkowalski May 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm, thats not ideal for people that just use 1 or 5 subscriptions... takes a long time...

how about something that works also well for the many use cases where people have less than 20 connections?

maybe there is a lib for throttling in golang or something we can build with the mutexes we used elsewhere?


if subscribed := m.publicClients[m.cid].SubAdded(sub); subscribed {
return m
}
Expand Down Expand Up @@ -286,7 +295,7 @@ func (m *Mux) addPublicClient() *Mux {
c, err := client.
New().
WithID(m.cid).
WithSubsLimit(20).
WithSubsLimit(pubClientSubsLimit).
Public(m.publicURL)
if err != nil {
m.Err = err
Expand Down