Changkun's Blog欧长坤的博客

Science and art, life in between.科学与艺术,生活在其间。

  • Home首页
  • Ideas想法
  • Posts文章
  • Tags标签
  • Bio关于
Changkun Ou

Changkun Ou

Human-AI interaction researcher, engineer, and writer.人机交互研究者、工程师、写作者。

Bridging HCI, AI, and systems programming. Building intelligent human-in-the-loop optimization systems. Informed by psychology, sociology, cognitive science, and philosophy.连接人机交互、AI 与系统编程。构建智能的人在环优化系统。融合心理学、社会学、认知科学与哲学。

Science and art, life in between.科学与艺术,生活在其间。

284 Blogs博客
173 Tags标签
Changkun's Blog欧长坤的博客
idea想法 2021-01-05 00:00:00

Daily Reading每日阅读

Read these articles:

  • The Right to Read. https://www.gnu.org/philosophy/right-to-read.en.html
  • Your Computer Isn’t Yours. https://sneak.berlin/20201112/your-computer-isnt-yours/
  • Pirate Cinema. https://craphound.com/pc/download/

Noticed these two clipboard-related projects:

  • https://github.com/binwiederhier/pcopy
  • https://github.com/nakabonne/pbgopy

It seems their development timelines are very close to when I developed midgard, but we all took very different paths:

  • pcopy focuses on the clipboard itself, with features like password protection, multiple clipboards, WebUI, etc.
  • pbgopy emphasizes security for cross-device syncing, with various key configurations, but very simple functionality — only supports text

The midgard I developed focuses on these features:

  • Automatic cross-device syncing (clipboard write-back), no commands needed
  • Ability to query devices currently online
  • Supports not only text but also image syncing
  • Supports creating public URLs for clipboard content
  • Supports converting code in clipboard to Carbon images
  • Supports keyboard shortcuts
  • …

读了这几篇文章:

  • 阅读的权利。https://www.gnu.org/philosophy/right-to-read.en.html
  • 你的电脑不属于你。https://sneak.berlin/20201112/your-computer-isnt-yours/
  • 盗版电影院。https://craphound.com/pc/download/

注意到了这两个跟剪贴板相关的项目:

  • https://github.com/binwiederhier/pcopy
  • https://github.com/nakabonne/pbgopy

看起来他们开发这两个项目的时间跟我开发 midgard 的时间非常接近,不过大家都走了很不同的路线:

  • pcopy 着重剪贴板本身,比如有剪贴板的密码保护,多种剪贴板、WebUI 等特性
  • pggopy 注重设备间同步的安全性,有各种密钥配置,但功能非常简单,只支持文本

我开发的 midgard 的则主打这些特性:

  • 多设备间自动同步(剪贴板回写),不需要敲命令
  • 能够查询同时在线的设备
  • 不仅支持文本,还支持图片的同步
  • 支持对剪贴板中的内容创建公开的 URL
  • 支持剪贴板中的代码转 Carbon 图片
  • 支持键盘快捷键
  • …

2020 Year-End Review2020 年终总结

Published at发布于:: 2021-01-03   |   Reading阅读:: 22 min

2020 is finally, completely over. As I have done every year since undergraduate, I open my blog and sit down to write this annual reflection. So much seemed to happen this year — I was still in the office working overtime on the very last day of it. If I had to name the single greatest takeaway from …

2020 年算是彻底结束了,像往常一样,我又打开了自己的博客,开始写下这篇我从本科开始就 坚持年更类型的文章。这一年里看似发生了许多,以至于在一年的最后一天我还在办公室里加班。 要说这一年里最大的收获是什么,可能会总结为两个字:转变。 到了博士阶段,也就不能像是本科、研究生那样只扩宽自己技能的广度,需要更多的专注在如何将自身 的能力做深度的培养(尽管学习的时候是系统性的学习,但如果没有长期实践这 …

Read More阅读更多 »
idea想法 2020-12-31 00:00:00

Daily Reading每日阅读

Read these articles:

  • A History of the GUI. http://www.cdpa.co.uk/UoP/Found/Downloads/reading6.pdf, https://www.readit-dtp.de/PDF/gui_history.pdf
  • History of the graphical user interface. https://en.wikipedia.org/wiki/History_of_the_graphical_user_interface
  • Algebraic Effects for the Rest of Us. https://overreacted.io/zh-hans/algebraic-effects-for-the-rest-of-us/
  • What does algebraic effects mean in FP? https://stackoverflow.com/questions/49626714/what-does-algebraic-effects-mean-in-fp
  • A Distributed Systems Reading List. https://dancres.github.io/Pages/
  • Tutte Embedding for Parameterization. https://isaacguan.github.io/2018/03/19/Tutte-Embedding-for-Parameterization/
  • Commentary on the Sixth Edition UNIX Operating System. http://www.lemis.com/grog/Documentation/Lions/index.php
  • Personal knowledge management beyond versioning. https://dl.acm.org/doi/10.1145/2362456.2362492
  • The Plain Text Life: Note Taking, Writing and Life Organization Using Plain Text Files. http://www.markwk.com/plain-text-life.html
  • Post-Evernote: How to Migrate Your Evernote Notes, Images and Tags Into Plain Text Markdown. http://www.markwk.com/migrate-evernote-plaintext.html
  • Five Levels of Error Handling in Both Python and JavaScript. https://dev.to/jesterxl/five-levels-of-error-handling-in-both-python-and-javascript-13ok

读了这些文章:

  • GUI 的历史。http://www.cdpa.co.uk/UoP/Found/Downloads/reading6.pdf, https://www.readit-dtp.de/PDF/gui_history.pdf
  • 图形用户界面的历史。https://en.wikipedia.org/wiki/History_of_the_graphical_user_interface
  • 写给普通人的代数效应。https://overreacted.io/zh-hans/algebraic-effects-for-the-rest-of-us/
  • 代数效应在函数式编程中意味着什么?https://stackoverflow.com/questions/49626714/what-does-algebraic-effects-mean-in-fp
  • 分布式系统阅读清单。https://dancres.github.io/Pages/
  • 用于参数化的 Tutte 嵌入。https://isaacguan.github.io/2018/03/19/Tutte-Embedding-for-Parameterization/
  • 第六版 UNIX 操作系统注释。http://www.lemis.com/grog/Documentation/Lions/index.php
  • 超越版本控制的个人知识管理。https://dl.acm.org/doi/10.1145/2362456.2362492
  • 纯文本生活:使用纯文本文件进行笔记、写作和生活管理。http://www.markwk.com/plain-text-life.html
  • 后 Evernote 时代:如何将 Evernote 笔记、图片和标签迁移到纯文本 Markdown。http://www.markwk.com/migrate-evernote-plaintext.html
  • Python 和 JavaScript 中错误处理的五个层次。https://dev.to/jesterxl/five-levels-of-error-handling-in-both-python-and-javascript-13ok
idea想法 2020-12-30 00:00:00

Daily Reading每日阅读

Read these articles:

  • The UNIXHATERS Handbook. http://web.mit.edu/~simsong/www/ugh.pdf
  • Why are video games graphics (still) a challenge? Productionizing rendering algorithms. https://bartwronski.com/2020/12/27/why-are-video-games-graphics-still-a-challenge-productionizing-rendering-algorithms/
  • BPF and Go: Modern forms of introspection in Linux. https://medium.com/bumble-tech/bpf-and-go-modern-forms-of-introspection-in-linux-6b9802682223
  • Systems design explains the world: volume 1. https://apenwarr.ca/log/20201227
  • Error handling guidelines for Go. https://jayconrod.com/posts/116/error-handling-guidelines-for-go
  • The Missing Semester of Your CS Education. https://missing.csail.mit.edu/
  • Build your own React. https://pomb.us/build-your-own-react/

读了这些文章:

  • UNIX 憎恨者手册。http://web.mit.edu/~simsong/www/ugh.pdf
  • 为什么电子游戏图形(仍然)是一个挑战?渲染算法的产品化。https://bartwronski.com/2020/12/27/why-are-video-games-graphics-still-a-challenge-productionizing-rendering-algorithms/
  • BPF 与 Go:Linux 中的现代内省方法。https://medium.com/bumble-tech/bpf-and-go-modern-forms-of-introspection-in-linux-6b9802682223
  • 系统设计解释世界:第一卷。https://apenwarr.ca/log/20201227
  • Go 错误处理指南。https://jayconrod.com/posts/116/error-handling-guidelines-for-go
  • 计算机科学教育中缺失的一课。https://missing.csail.mit.edu/
  • 构建你自己的 React。https://pomb.us/build-your-own-react/

Migration with Zero Downtime零宕机迁移

Published at发布于:: 2020-12-28   |   Reading阅读:: 12 min

In this post I gave an overview of the restructured architecture of changkun.de, but I didn’t go into detail about how the migration itself was carried out — whether there was any downtime, and so on. This time, let’s take a closer look at the migration process. Migration Checklist …

在这篇文章 中我介绍了 changkun.de 重新调整后的一个整体的架构, 但并没有仔细的介绍进行架构升级的过程是怎样的、升级过程中是否有进行停机等等。 这次我们就来简单聊一聊这个迁移过程。 迁移清单 首先,changkun.de 服务器上运行了诸多服务,比如使用频繁的有 redir、midgard 等等,如图所示: 这些服务包括: https://changkun.de/s/main …

Read More阅读更多 »
idea想法 2020-12-27 00:00:00

Concurrency Patterns并发模式

Fan-In multiplexes multiple input channels onto one output channel.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
func Funnel(sources ...<-chan int) <-chan int {
    dest := make(chan int)                  // The shared output channel

    var wg sync.WaitGroup                   // Used to automatically close dest
                                            // when all sources are closed

    wg.Add(len(sources))                    // Increment the sync.WaitGroup

    for _, ch := range sources {            // Start a goroutine for each source
        go func(c <-chan int) {
            defer wg.Done()                 // Notify WaitGroup when c closes

            for n := range c {
                dest <- n
            }
        }(ch)
    }

    go func() {                             // Start a goroutine to close dest
        wg.Wait()                           // after all sources close
        close(dest)
    }()

    return dest
}

Fan-Out evenly distributes messages from an input channel to multiple output channels.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
func Split(source <-chan int, n int) []<-chan int {
    dests := make([]<-chan int, 0)          // Create the dests slice

    for i := 0; i < n; i++ {                // Create n destination channels
        ch := make(chan int)
        dests = append(dests, ch)

        go func() {                         // Each channel gets a dedicated
            defer close(ch)                 // goroutine that competes for reads

            for val := range source {
                ch <- val
            }
        }()
    }

    return dests
}

Future provides a placeholder for a value that’s not yet known.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
type Future interface {
    Result() (string, error)
}

type InnerFuture struct {
    once sync.Once
    wg   sync.WaitGroup

    res   string
    err   error
    resCh <-chan string
    errCh <-chan error
}

func (f *InnerFuture) Result() (string, error) {
    f.once.Do(func() {
        f.wg.Add(1)
        defer f.wg.Done()
        f.res = <-f.resCh
        f.err = <-f.errCh
    })

    f.wg.Wait()

    return f.res, f.err
}

func SlowFunction(ctx context.Context) Future {
    resCh := make(chan string)
    errCh := make(chan error)

    go func() {
        select {
        case <-time.After(time.Second * 2):
            resCh <- "I slept for 2 seconds"
            errCh <- nil
        case <-ctx.Done():
            resCh <- ""
            errCh <- ctx.Err()
        }
    }()

    return &InnerFuture{resCh: resCh, errCh: errCh}
}

Sharding splits a large data structure into multiple partitions to localize the effects of read/write locks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
type Shard struct {
    sync.RWMutex                            // Compose from sync.RWMutex
    m map[string]interface{}                // m contains the shard's data
}

type ShardedMap []*Shard                    // ShardedMap is a *Shards slice

func NewShardedMap(nshards int) ShardedMap {
    shards := make([]*Shard, nshards)       // Initialize a *Shards slice

    for i := 0; i < nshards; i++ {
        shard := make(map[string]interface{})
        shards[i] = &Shard{m: shard}
    }

    return shards                           // A ShardedMap IS a *Shards slice!
}

func (m ShardedMap) getShardIndex(key string) int {
    checksum := sha1.Sum([]byte(key))       // Use Sum from "crypto/sha1"
    hash := int(checksum[17])               // Pick a random byte as our hash
    index := hash % len(shards)             // Mod by len(shards) to get index
}

func (m ShardedMap) getShard(key string) *Shard {
    index := m.getShardIndex(key)
    return m[index]
}

func (m ShardedMap) Get(key string) interface{} {
    shard := m.getShard(key)
    shard.RLock()
    defer shard.RUnlock()

    return shard.m[key]
}

func (m ShardedMap) Set(key string, value interface{}) {
    shard := m.getShard(key)
    shard.Lock()
    defer shard.Unlock()

    shard.m[key] = value
}

func (m ShardedMap) Keys() []string {
    keys := make([]string, 0)               // Create an empty keys slice
    wg := sync.WaitGroup{}                  // Create a wait group and add a
    wg.Add(len(m))                          // wait value for each slice
    for _, shard := range m {               // Run a goroutine for each slice
        go func(s *Shard) {
            s.RLock()                       // Establish a read lock on s
            for key, _ := range s.m {       // Get the slice's keys
                keys = append(keys, key)
            }
            s.RUnlock()                     // Release the read lock
            wg.Done()                       // Tell the WaitGroup it's done
        }(shard)
    }
    wg.Wait()                               // Block until all reads are done
    return keys                             // Return combined keys slice
}

扇入(Fan-In) 将多个输入通道多路复用到一个输出通道上。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
func Funnel(sources ...<-chan int) <-chan int {
    dest := make(chan int)                  // The shared output channel

    var wg sync.WaitGroup                   // Used to automatically close dest
                                            // when all sources are closed

    wg.Add(len(sources))                    // Increment the sync.WaitGroup

    for _, ch := range sources {            // Start a goroutine for each source
        go func(c <-chan int) {
            defer wg.Done()                 // Notify WaitGroup when c closes

            for n := range c {
                dest <- n
            }
        }(ch)
    }

    go func() {                             // Start a goroutine to close dest
        wg.Wait()                           // after all sources close
        close(dest)
    }()

    return dest
}

扇出(Fan-Out) 将输入通道的消息均匀分发到多个输出通道。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
func Split(source <-chan int, n int) []<-chan int {
    dests := make([]<-chan int, 0)          // Create the dests slice

    for i := 0; i < n; i++ {                // Create n destination channels
        ch := make(chan int)
        dests = append(dests, ch)

        go func() {                         // Each channel gets a dedicated
            defer close(ch)                 // goroutine that competes for reads

            for val := range source {
                ch <- val
            }
        }()
    }

    return dests
}

Future 为尚未确定的值提供一个占位符。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
type Future interface {
    Result() (string, error)
}

type InnerFuture struct {
    once sync.Once
    wg   sync.WaitGroup

    res   string
    err   error
    resCh <-chan string
    errCh <-chan error
}

func (f *InnerFuture) Result() (string, error) {
    f.once.Do(func() {
        f.wg.Add(1)
        defer f.wg.Done()
        f.res = <-f.resCh
        f.err = <-f.errCh
    })

    f.wg.Wait()

    return f.res, f.err
}

func SlowFunction(ctx context.Context) Future {
    resCh := make(chan string)
    errCh := make(chan error)

    go func() {
        select {
        case <-time.After(time.Second * 2):
            resCh <- "I slept for 2 seconds"
            errCh <- nil
        case <-ctx.Done():
            resCh <- ""
            errCh <- ctx.Err()
        }
    }()

    return &InnerFuture{resCh: resCh, errCh: errCh}
}

分片(Sharding) 将大型数据结构拆分为多个分区,以局部化读写锁的影响。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
type Shard struct {
    sync.RWMutex                            // Compose from sync.RWMutex
    m map[string]interface{}                // m contains the shard's data
}

type ShardedMap []*Shard                    // ShardedMap is a *Shards slice

func NewShardedMap(nshards int) ShardedMap {
    shards := make([]*Shard, nshards)       // Initialize a *Shards slice

    for i := 0; i < nshards; i++ {
        shard := make(map[string]interface{})
        shards[i] = &Shard{m: shard}
    }

    return shards                           // A ShardedMap IS a *Shards slice!
}

func (m ShardedMap) getShardIndex(key string) int {
    checksum := sha1.Sum([]byte(key))       // Use Sum from "crypto/sha1"
    hash := int(checksum[17])               // Pick a random byte as our hash
    index := hash % len(shards)             // Mod by len(shards) to get index
}

func (m ShardedMap) getShard(key string) *Shard {
    index := m.getShardIndex(key)
    return m[index]
}

func (m ShardedMap) Get(key string) interface{} {
    shard := m.getShard(key)
    shard.RLock()
    defer shard.RUnlock()

    return shard.m[key]
}

func (m ShardedMap) Set(key string, value interface{}) {
    shard := m.getShard(key)
    shard.Lock()
    defer shard.Unlock()

    shard.m[key] = value
}

func (m ShardedMap) Keys() []string {
    keys := make([]string, 0)               // Create an empty keys slice
    wg := sync.WaitGroup{}                  // Create a wait group and add a
    wg.Add(len(m))                          // wait value for each slice
    for _, shard := range m {               // Run a goroutine for each slice
        go func(s *Shard) {
            s.RLock()                       // Establish a read lock on s
            for key, _ := range s.m {       // Get the slice's keys
                keys = append(keys, key)
            }
            s.RUnlock()                     // Release the read lock
            wg.Done()                       // Tell the WaitGroup it's done
        }(shard)
    }
    wg.Wait()                               // Block until all reads are done
    return keys                             // Return combined keys slice
}
idea想法 2020-12-26 00:00:00

Stability Patterns稳定性模式

Circuit Breaker automatically degrades service functions in response to a likely fault, preventing larger or cascading failures by eliminating recurring errors and providing reasonable error responses.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
type Circuit func(context.Context) (string, error)

// Breaker wraps a circuit with a given failure threashould.
func Breaker(circuit Circuit, failureThreshold uint64) Circuit {
    var lastStateSuccessful = true
    var consecutiveFailures uint64 = 0
    var lastAttempt time.Time = time.Now()

    return func(ctx context.Context) (string, error) {
        if consecutiveFailures >= failureThreshold {
            backoffLevel := consecutiveFailures - failureThreshold
            shouldRetryAt := lastAttempt.Add(time.Second * 2 << backoffLevel)

            if !time.Now().After(shouldRetryAt) {
                return "", errors.New("circuit open -- service unreachable")
            }
        }

        lastAttempt = time.Now()
        response, err := circuit(ctx)

        if err != nil {
            if !lastStateSuccessful {
                consecutiveFailures++
            }
            lastStateSuccessful = false
            return response, err
        }

        lastStateSuccessful = true
        consecutiveFailures = 0

        return response, nil
    }
}

Debounce limits the frequency of a function call to one among a cluster of invocations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
type Circuit func(context.Context) (string, error)

func DebounceFirst(circuit Circuit, d time.Duration) Circuit {
    var threshold time.Time
    var cResult string
    var cError error

    return func(ctx context.Context) (string, error) {
        if threshold.Before(time.Now()) {
            cResult, cError = circuit(ctx)
        }

        threshold = time.Now().Add(d)
        return cResult, cError
    }
}
func DebounceLast(circuit Circuit, d time.Duration) Circuit {
    var threshold time.Time = time.Now()
    var ticker *time.Ticker
    var result string
    var err error

    return func(ctx context.Context) (string, error) {
        threshold = time.Now().Add(d)

        if ticker == nil {
            ticker = time.NewTicker(time.Millisecond * 100)
            tickerc := ticker.C

            go func() {
                defer ticker.Stop()

                for {
                    select {
                    case <-tickerc:
                        if threshold.Before(time.Now()) {
                            result, err = circuit(ctx)
                            ticker.Stop()
                            ticker = nil
                            break
                        }
                    case <-ctx.Done():
                        result, err = "", ctx.Err()
                        break
                    }
                }
            }()
        }

        return result, err
    }
}

Retry accounts for a possible transient fault in a distributed system by transparently retrying a failed operation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
type Effector func(context.Context) (string, error)

func Retry(effector Effector, retries int, delay time.Duration) Effector {
    return func(ctx context.Context) (string, error) {
        for r := 0; ; r++ {
            response, err := effector(ctx)
            if err == nil || r >= retries {
                return response, err
            }

            log.Printf("Attempt %d failed; retrying in %v", r + 1, delay)

            select {
            case <-time.After(delay):
            case <-ctx.Done():
                return "", ctx.Err()
            }
        }
    }
}

Throttle limits the frequency of a function call to some maximum number of invocations per unit of time.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
type Effector func(context.Context) (string, error)

func Throttle(e Effector, max uint, refill uint, d time.Duration) Effector {
    var ticker *time.Ticker = nil
    var tokens uint = max

    var lastReturnString string
    var lastReturnError error

    return func(ctx context.Context) (string, error) {
        if ctx.Err() != nil {
            return "", ctx.Err()
        }

        if ticker == nil {
            ticker = time.NewTicker(d)
            defer ticker.Stop()

            go func() {
                for {
                    select {
                    case <-ticker.C:
                        t := tokens + refill
                        if t > max {
                            t = max
                        }
                        tokens = t
                    case <-ctx.Done():
                        ticker.Stop()
                        break
                    }
                }
            }()
        }

        if tokens > 0 {
            tokens--
            lastReturnString, lastReturnError = e(ctx)
        }

        return lastReturnString, lastReturnError
    }
}

Timeout allows a process to stop waiting for an answer once it’s clear that an answer may not be coming.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
func Timeout(slowfunc func(interface{}) (interface{}, error),
    arg interface{}) func(context.Context) (interface{}, error) {
    chres := make(chan interface{})
    cherr := make(chan error)

    go func() {
        res, err := flowfunc(arg)
        chres <- res
        cherr <- err
    }()

    return func(ctx context.Context) (interface{}, error) {
        select {
        case res := <-chres:
            return res, <-cherr
        case <-ctx.Done():
            return "", ctx.Err()
        }
    }
}

断路器(Circuit Breaker) 在可能发生故障时自动降级服务功能,通过消除重复错误并提供合理的错误响应来防止更大的或级联的故障。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
type Circuit func(context.Context) (string, error)

// Breaker wraps a circuit with a given failure threashould.
func Breaker(circuit Circuit, failureThreshold uint64) Circuit {
    var lastStateSuccessful = true
    var consecutiveFailures uint64 = 0
    var lastAttempt time.Time = time.Now()

    return func(ctx context.Context) (string, error) {
        if consecutiveFailures >= failureThreshold {
            backoffLevel := consecutiveFailures - failureThreshold
            shouldRetryAt := lastAttempt.Add(time.Second * 2 << backoffLevel)

            if !time.Now().After(shouldRetryAt) {
                return "", errors.New("circuit open -- service unreachable")
            }
        }

        lastAttempt = time.Now()
        response, err := circuit(ctx)

        if err != nil {
            if !lastStateSuccessful {
                consecutiveFailures++
            }
            lastStateSuccessful = false
            return response, err
        }

        lastStateSuccessful = true
        consecutiveFailures = 0

        return response, nil
    }
}

防抖(Debounce) 将一组调用中的函数调用频率限制为仅执行一次。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
type Circuit func(context.Context) (string, error)

func DebounceFirst(circuit Circuit, d time.Duration) Circuit {
    var threshold time.Time
    var cResult string
    var cError error

    return func(ctx context.Context) (string, error) {
        if threshold.Before(time.Now()) {
            cResult, cError = circuit(ctx)
        }

        threshold = time.Now().Add(d)
        return cResult, cError
    }
}
func DebounceLast(circuit Circuit, d time.Duration) Circuit {
    var threshold time.Time = time.Now()
    var ticker *time.Ticker
    var result string
    var err error

    return func(ctx context.Context) (string, error) {
        threshold = time.Now().Add(d)

        if ticker == nil {
            ticker = time.NewTicker(time.Millisecond * 100)
            tickerc := ticker.C

            go func() {
                defer ticker.Stop()

                for {
                    select {
                    case <-tickerc:
                        if threshold.Before(time.Now()) {
                            result, err = circuit(ctx)
                            ticker.Stop()
                            ticker = nil
                            break
                        }
                    case <-ctx.Done():
                        result, err = "", ctx.Err()
                        break
                    }
                }
            }()
        }

        return result, err
    }
}

重试(Retry) 通过透明地重试失败的操作来处理分布式系统中可能出现的瞬时故障。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
type Effector func(context.Context) (string, error)

func Retry(effector Effector, retries int, delay time.Duration) Effector {
    return func(ctx context.Context) (string, error) {
        for r := 0; ; r++ {
            response, err := effector(ctx)
            if err == nil || r >= retries {
                return response, err
            }

            log.Printf("Attempt %d failed; retrying in %v", r + 1, delay)

            select {
            case <-time.After(delay):
            case <-ctx.Done():
                return "", ctx.Err()
            }
        }
    }
}

节流(Throttle) 将函数调用的频率限制在每单位时间内的最大调用次数。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
type Effector func(context.Context) (string, error)

func Throttle(e Effector, max uint, refill uint, d time.Duration) Effector {
    var ticker *time.Ticker = nil
    var tokens uint = max

    var lastReturnString string
    var lastReturnError error

    return func(ctx context.Context) (string, error) {
        if ctx.Err() != nil {
            return "", ctx.Err()
        }

        if ticker == nil {
            ticker = time.NewTicker(d)
            defer ticker.Stop()

            go func() {
                for {
                    select {
                    case <-ticker.C:
                        t := tokens + refill
                        if t > max {
                            t = max
                        }
                        tokens = t
                    case <-ctx.Done():
                        ticker.Stop()
                        break
                    }
                }
            }()
        }

        if tokens > 0 {
            tokens--
            lastReturnString, lastReturnError = e(ctx)
        }

        return lastReturnString, lastReturnError
    }
}

超时(Timeout) 允许进程在明确答案可能不会到来时停止等待。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
func Timeout(slowfunc func(interface{}) (interface{}, error),
    arg interface{}) func(context.Context) (interface{}, error) {
    chres := make(chan interface{})
    cherr := make(chan error)

    go func() {
        res, err := flowfunc(arg)
        chres <- res
        cherr <- err
    }()

    return func(ctx context.Context) (interface{}, error) {
        select {
        case res := <-chres:
            return res, <-cherr
        case <-ctx.Done():
            return "", ctx.Err()
        }
    }
}

2020 Reading List2020 读书清单

Published at发布于:: 2020-12-25   |   Reading阅读:: 4 min

It&rsquo;s the end of the year again — time to compile this year&rsquo;s reading list. 2020 was a truly extraordinary year, and I didn&rsquo;t read all that many books. Most were related to my field, and several were rereads from undergrad. Those classics always yield something new on each reading. …

又到年末了,是时候整理一波今年的读书清单了。2020 年实在是太特殊了,这一年里读过的书其实也不多, 大多与专业相关,且好几本是本科时期读过再来重读的,这类书过于经典,每次读的收获都会有所不同。 人文类 今年人文类的书籍其实集中在了理解上个世纪计算机的发展史上了,包括软件的部分包括 UNIX、 Linux、自由软件运动等,硬件则包括微计算机的兴衰。 所以今年花了一些时间去读这些书,按推荐强弱先后 …

Read More阅读更多 »
idea想法 2020-12-25 00:00:00

LBRYLBRY

A decentralized video platform – could it be a competitor to YouTube?

https://odysee.com/

一个去中心化的视频平台,似乎是 YouTube 的竞争对手?

https://odysee.com/

The All in Go Stack全面转向 Go 技术栈

Published at发布于:: 2020-12-24   |   Reading阅读:: 26 min

Since graduating from university I have had very little time to tinker with my own website. Most of the time I would ssh into the server, throw together some service, and then leave it to fend for itself — even if a service went down I did not pay much attention. After all, it was something I ran …

本科毕业以后就很少有时间来折腾自己的网站了,大部分时间里都是 ssh 到服务器上随便搭 好一个服务后就任其自由生长,即便是服务挂掉了之后也没有太在意——毕竟是自己用的东西, 对于网站的用户而言:ABSOLUTELY NO WARRANTY。不过随着网站近来的用户数量呈现难以置信 的增长势头(虽然什么也没干),自然也就难免希望让网站变得更加「可靠」。 作为 2020 年的最后几项 TODO,我终于在 …

Read More阅读更多 »
4 5 6 7 8 9 10 11 12
© 2008 - 2026 Changkun Ou. All rights reserved.保留所有权利。 | PV/UV: /
0%