Read this article:

• What are the most important statistical ideas of the past 50 years? https://arxiv.org/pdf/2012.00174.pdf

Runtime changes:

• New GC Pacer: https://golang.org/issue/44167
• Scheduler performance: https://golang.org/issue/43997

Toolchain changes:

• Register-based calling convention: https://golang.org/issue/40724
• Fuzzing: golang.org/s/draft-fuzzing-design

Today, the Go team released a brand new GC pacer design. Let’s briefly discuss what problems existed in the previous design and what the new design aims to solve.

The current Go runtime GC is a concurrent mark-sweep collector, which involves two core problems that need to be solved: 1) when to start GC and how many workers to launch for collection, to prevent the collector from using too many computing resources and affecting efficient execution of user code; 2) how to prevent the garbage collection speed from being slower than the memory allocation speed.

To address these problems, as early as Go 1.5, the Go team treated this as an optimization problem of minimizing heap growth rate and CPU usage, which led to two key components: 1) the pacer: predicting GC trigger timing based on heap growth speed; 2) mark assist: pausing user code that allocates too fast, redirecting goroutines that are allocating memory to perform garbage marking work, in order to smoothly complete the current GC cycle.

However, when making pacing decisions, this GC contains a hidden assumption: the allocation rate is always a constant (1+GOGC/100). Unfortunately, due to the existence of mark assist and discrepancies between implementation and the theoretical model, this assumption is actually incorrect. This leads to several hard-to-solve problems: 1) when the allocation rate violates the constant assumption, the predicted start time is too late, requiring excessive CPU consumption — while GOGC can be dynamically adjusted, it remains a hyperparameter requiring extensive domain experience to tune manually; 2) since the optimization targets heap growth without heap memory size limits, either setting GOGC too large or encountering peak allocations causes rapid heap growth leading to OOM; 3) newly allocated memory within the current GC cycle is left to the next GC cycle for collection, and mark assist’s allocation throttling causes latency pauses (STW); 4) …

So what has the new pacer redesigned to solve these problems?

As mentioned above, the main source of various problems is the incorrect assumption that the allocation rate is a constant. So naturally, it’s easy to think of using the mark assist component to dynamically calculate the allocation rate during modeling, thereby achieving the goal of dynamically adjusting the heap target. Unfortunately, the original design only tracked allocations on the heap, without considering the stack or global variables. To make the problem more comprehensive, the new design introduces an “assist ratio” — the ratio of allocations produced but not collected in the current GC cycle (A) to the amount of scanning completed in the current GC cycle (B), i.e., A/B. This metric more intuitively reflects the actual difficulty of GC work: if the user allocation rate is too high, A increases, the assist ratio rises, and more assistance is needed from the mark assist; if the allocation rate is moderate, the assist ratio decreases. With the introduction of the assist ratio, the pacer can dynamically adjust the assist work, thereby resolving the pauses caused by the assist.

Let’s look at a practical scenario: when a sudden burst of peak requests arrives, the number of goroutines increases dramatically, generating a large number of stacks and allocation tasks. Here are the simulation results: Figure 1 shows the pacer before adjustment, Figure 2 shows the pacer after adjustment. As shown in the lower-left of Figure 1, the heap target workload is consistently underestimated, causing the heap to always overshoot; while the new pacer can quickly converge to zero and complete the heap target prediction. The upper-right of Figure 1 shows that the actual GC CPU usage is always lower than the target usage, failing to meet the expected metrics; while the newly designed pacer can quickly converge to the target CPU usage.

Of course, due to space constraints, the above is only a very brief introduction to the new pacer design. If you are interested in this topic, you can refer to the following links. There will be opportunities to share more detailed analysis in the future.

1. Existing problems with the GC pacer: https://golang.org/issue/42430
2. Design document for the new pacer: https://go.googlesource.com/proposal/+/a216b56e743c5b6b300b3ef1673ee62684b5b63b/design/44167-gc-pacer-redesign.md
3. Related proposal: https://golang.org/issue/44167
4. Simulator for the new GC pacer model: https://github.com/mknyszek/pacer-model

Go 1.16 has released many very interesting changes. Here is a brief summary:

russ cos: deprecated.

• https://twitter.com/_rsc/status/1351676094664110082
• https://go-review.googlesource.com/c/go/+/285378
• https://github.com/golang/go/issues/43724

1. Support for darwin/arm64
   1. Issues encountered supporting darwin/arm64
      • Apple’s bug: related to signal preemption
   2. Apple Silicon M1 performance
      • But crypto performance is poor
      • Release cycle: https://github.com/golang/go/wiki/Go-Release-Cycle
   3. Compiler bootstrapping process

• Installing Go: https://gist.github.com/Dids/dbe6356377e2a0b0dc8eacb0101dc3a7

• https://github.com/golang/go/issues/42684

  • Kernel Panic Episode 62: Your Computer Isn’t Yours, Code Signing, OCSP Server
  • Ken Thompson Turing Award lecture: Reflections on Trusting Trust
    • TODO
  • Apple’s long-standing code signing issues; I encountered similar problems doing Electron in the early days, and these issues still exist today

• Asynchronous preemption random crashes, a Rosetta bug: https://github.com/golang/go/issues/42700

• Bootstrapping, installation confusion: https://github.com/golang/go/issues/38485#issuecomment-735360572

  • Go’s bootstrapping consists of three steps
    • 0. 1.4 C version TODO
    • 1. tool chain 1
    • 2. tool chain 2
    • 3. tool chain 3

• Run x86 programs under Rosetta: arch --x86_64

• M1 compatibility status in dotfiles: https://github.com/changkun/dotfiles/issues/2

  • https://doesitarm.com/
  • https://isapplesiliconready.com/

• Got it in early December, have been using it for almost two months now — very smooth, battery life is incredible

• My essential third-party software list:

  • homebrew (compatibility is not great, but fortunately most dependent software is written in Go, and Go’s support is very complete)
    • Breaks compatibility casually, removes software distribution — there was a tool called rmtrash that I had been using since around 2014, but it was removed from distribution last year, so I wrote a fully compatible tool changkun.de/s/rmtrash, but it wasn’t merged; they said it needed to be approved by the original author to bypass popularity restrictions, but the original author is unreachable
  • vscode (have been using Insider long-term)
  • macvim
  • tmux
  • oh-my-zsh
  • Blender (Cycles ray tracing doesn’t support GPU, but editing meshes with less than a million vertices is fine)
  • iTerm: supports M1
  • Chrome: supports M1
  • MacTex: supports M1
  • Docker: released support a week before Christmas, works perfectly, no issues so far

1. Go Modules changes

   1. Collecting feedback
   2. Complex dependency management — what’s the most complex project dependency you’ve managed in practice, how many modules, and what do you write for each dependency upgrade? What did you use before Go modules?
      1. My experience: Go vendor, 1.10 dep, 1.11 go modules
      2. GOPATH project management — although GOPATH has been removed, I still follow the GOPATH convention
   3. Minimum version selection
      1. Semantic Versioning: major.minor.patch
      2. The classic diamond dependency problem: A depends on B and C, B and C each depend on different versions of D that are incompatible, so no specific version of D can be selected — semantic import versioning eliminates this by adding the major version number requirement at the end of the import path /v2
      3. dep doesn’t allow diamond dependencies, upgrades are very difficult
      4. Build reproducibility — without a lock file, >= dependencies change over time
      5. Select the minimum usable version — builds don’t change over time
      6. https://www.youtube.com/watch?v=F8nrpe0XWRg&ab_channel=SingaporeGophers
      7. Misunderstood working methods
      8. GOPATH
      9. vendor
      10. Three key points
      11. Compatibility
      12. Reproducibility
      13. Cooperation (often overlooked by many)
   4. Go Modules enabled by default, go build must include go.mod file, otherwise compilation fails
   5. build/test will not upgrade modules
   6. Default -mod=vendor

2. File system interface

   1. Why is the fs.FS abstraction important

      1. Unix file system abstract always disk blocks
      2. Network file systems (Upspin) abstract away machines
      3. REST abstracts nearly anything
      4. cp doesn’t care whether it’s moving file blocks, or even where the file is — it could be different disks or different machines
      5. Defines the “generics” for any file type tools

   2. What major changes it caused

      1. io/ioutil
         1. Russ Cox’s explanation of deprecated in Go (https://twitter.com/_rsc/status/1351676094664110082)
         2. https://www.srcbeat.com/2021/01/golang-ioutil-deprecated/
      2. Other fs abstractions
      3. Rob Pike’s 2016/2017 Gopherfest, Upspin, Changkun’s Midgard
         1. https://www.youtube.com/watch?v=ENLWEfi0Tkg&ab_channel=TheGoProgrammingLanguage
         2. FUSE: filesystem in userspace
         3. https://changkun.de/s/midgard
         4. Every user has a private root, no global root, r@golang.org/some/stuff, user names look like email addresses
         5. Access control defined by plain text files read: r@golang.org, ann@example.com
      4. Currently a very simple implementation, just a read-only file system
      5. ReadDir and DirEntry
         1. https://benhoyt.com/writings/go-readdir/
      6. Extensible directions: memoryFS, support for writing back to disk, hashFS for CDN support
      7. Remaining issues… e.g. 44166

      1 2 3 4 5 6 7

      import _ "embed" //go:embed a.txt var s string import "embed" type embed.String string var s embed.String

      1

3. File embedding //go:embed

   1. Basic functionality of the new feature
   2. Some possible applications
   3. Some features discussed during the feature freeze cycle
   4. https://blog.carlmjohnson.net/post/2021/how-to-use-go-embed/

4. Runtime memory management

   1. Return to MADV_DONTNEED
   • https://blog.changkun.de/posts/pss-uss-rss/
   2. New monitoring infrastructure runtime/metrics
   • Previous monitoring functions: runtime.ReadMemStats, debug.GCStats
   • runtime/metrics:
     • metrics.All()
     • Issue 37112

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

package main

import (
	"fmt"
	"runtime/metrics"
)

func main() {
	// Get descriptions for all supported metrics.
	descs := metrics.All()

	// Create a sample for each metric.
	samples := make([]metrics.Sample, len(descs))
	for i := range samples {
		samples[i].Name = descs[i].Name
	}

	// Sample the metrics. Re-use the samples slice if you can!
	metrics.Read(samples)

	// Iterate over all results.
	for _, sample := range samples {
		// Pull out the name and value.
		name, value := sample.Name, sample.Value

		// Handle each sample.
		switch value.Kind() {
		case metrics.KindUint64:
			fmt.Printf("%s: %d\n", name, value.Uint64())
		case metrics.KindFloat64:
			fmt.Printf("%s: %f\n", name, value.Float64())
		case metrics.KindFloat64Histogram:
			// The histogram may be quite large, so let's just pull out
			// a crude estimate for the median for the sake of this example.
			fmt.Printf("%s: %f\n", name, medianBucket(value.Float64Histogram()))
		case metrics.KindBad:
			// This should never happen because all metrics are supported
			// by construction.
			panic("bug in runtime/metrics package!")
		default:
			// This may happen as new metrics get added.
			//
			// The safest thing to do here is to simply log it somewhere
			// as something to look into, but ignore it for now.
			// In the worst case, you might temporarily miss out on a new metric.
			fmt.Printf("%s: unexpected metric Kind: %v\n", name, value.Kind())
		}
	}
}

func medianBucket(h *metrics.Float64Histogram) float64 {
	total := uint64(0)
	for _, count := range h.Counts {
		total += count
	}
	thresh := total / 2
	total = 0
	for i, count := range h.Counts {
		total += count
		if total > thresh {
			return h.Buckets[i]
		}
	}
	panic("should not happen")
}

1. Other noteworthy features
   1. os/signal.NotifyContext
   2. Memory model fixes
   3. Linker optimizations

Read these articles:

• What to expect when monitoring memory usage for modern Go applications. https://www.bwplotka.dev/2019/golang-memory-monitoring/
• Distributed Systems. https://www.youtube.com/watch?v=UEAMfLPZZhE
• Golang News. https://www.golangnews.com/
• SIGCHI Symposium on Engineering Interactive Computing Systems. http://eics.acm.org/
• runtime: use MADV_FREE on Linux if available. https://go-review.googlesource.com/c/go/+/135395/
• runtime: make the page allocator scale. https://github.com/golang/go/issues/35112
• runtime: add per-p mspan cache. https://go-review.googlesource.com/c/go/+/196642
• A New Smoothing Algorithm for Quadrilateral and Hexahedral Meshes. https://link.springer.com/content/pdf/10.1007%2F11758525_32.pdf
• OpenGL Docs. https://docs.gl
• On Playing Chess. https://blog.gardeviance.org/2018/03/on-playing-chess.html
• Memory Models: A Case For Rethinking Parallel Languages and Hardware. https://cacm.acm.org/magazines/2010/8/96610-memory-models-a-case-for-rethinking-parallel-languages-and-hardware/fulltext
• Engineer level & competency framework. https://github.com/spring2go/engineer_competency_framework
• A Concurrent Window System. http://doc.cat-v.org/bell_labs/concurrent_window_system/concurrent_window_system.pdf

How to create a window using Go? macOS has Cocoa, Linux has X11, but accessing these APIs seems to require Cgo. Is it possible to avoid Cgo? Some existing GUI libraries and graphics engines:

GUI Toolkits:

• https://github.com/hajimehoshi/ebiten
• https://github.com/gioui/gio
• https://github.com/fyne-io/fyne
• https://github.com/g3n/engine
• https://github.com/goki/gi
• https://github.com/peterhellberg/gfx
• https://golang.org/x/exp/shiny

2D/3D Graphics:

• https://github.com/llgcode/draw2d
• https://github.com/fogleman/gg
• https://github.com/ajstarks/svgo
• https://github.com/BurntSushi/graphics-go
• https://github.com/azul3d/engine
• https://github.com/KorokEngine/Korok
• https://github.com/EngoEngine/engo/
• http://mumax.github.io/

Here is a partial list:

https://github.com/avelino/awesome-go#gui

Most people use glfw and OpenGL. Here are some required libraries (Cgo bindings):

• https://github.com/go-gl/gl
• https://github.com/go-gl/glfw
• https://github.com/remogatto/egl

Some more low-level ones, e.g. X-related:

• X bindings: https://github.com/BurntSushi/xgb
• X window management: https://github.com/BurntSushi/wingo

For example, if you need Metal on macOS:

• https://dmitri.shuralyov.com/gpu/mtl

Like the GUI tool ebiten mentioned above, on Windows it no longer needs Cgo. The approach seems to be packaging the window management DLLs directly into the binary and then using DLL dynamic linking calls.

Besides GLFW, there is the heavier SDL:

• https://github.com/veandco/go-sdl2

Relationships between some basic terms:

1
2
3
4
5
6
7

Unix:      The ancestor
BSD:       Unix-like, Berkeley distribution, one of the two traditional flavors of Unix
System V:  Developed by AT&T, one of the two traditional flavors of Unix
Linux:     A fresh flavor of Unix-like OS
POSIX:     A standard attempting to reduce implementation differences
Darwin:    Apple's open-source Unix-like kernel XNU
Mach-O:    Microkernel hybridized in Darwin, developed by CMU

By Eraserhead1, Infinity0, Sav_vas - Levenez Unix History Diagram, Information on the history of IBM's AIX on ibm.com, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1801948

Relationships between some Wayland-related tools:

1
2
3
4
5
6

X Window System == X11: Display standard developed by MIT, the foundation for GNOME and KDE
Xorg:    The official implementation of X11
Wayland: An alternative communication protocol between display server and client, distinct from X11
EGL:     Wayland clients use EGL to directly manipulate the framebuffer
libDRM:  Kernel API exposed to userspace, underlying dependency of EGL/X11
DRM:     Kernel-level component for manipulating the framebuffer

By Shmuel Csaba Otto Traian, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=28029855

By Shmuel Csaba Otto Traian, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=31768083

By Shmuel Csaba Otto Traian, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=27858390

By Shmuel Csaba Otto Traian，CC BY-SA 3.0，https://commons.wikimedia.org/w/index.php?curid=27799196

So through DRM you can directly operate on the Frame Buffer under Linux, i.e., Direct Rendering Manager. Related libraries:

• https://github.com/NeowayLabs/drm

With this, you can do pure Go rendering directly on Linux, eliminating the dependency on C legacy.

Read these articles:

• The Right to Read. https://www.gnu.org/philosophy/right-to-read.en.html
• Your Computer Isn’t Yours. https://sneak.berlin/20201112/your-computer-isnt-yours/
• Pirate Cinema. https://craphound.com/pc/download/

Noticed these two clipboard-related projects:

• https://github.com/binwiederhier/pcopy
• https://github.com/nakabonne/pbgopy

It seems their development timelines are very close to when I developed midgard, but we all took very different paths:

• pcopy focuses on the clipboard itself, with features like password protection, multiple clipboards, WebUI, etc.
• pbgopy emphasizes security for cross-device syncing, with various key configurations, but very simple functionality — only supports text

The midgard I developed focuses on these features:

• Automatic cross-device syncing (clipboard write-back), no commands needed
• Ability to query devices currently online
• Supports not only text but also image syncing
• Supports creating public URLs for clipboard content
• Supports converting code in clipboard to Carbon images
• Supports keyboard shortcuts
• …

Read these articles:

• A History of the GUI. http://www.cdpa.co.uk/UoP/Found/Downloads/reading6.pdf, https://www.readit-dtp.de/PDF/gui_history.pdf
• History of the graphical user interface. https://en.wikipedia.org/wiki/History_of_the_graphical_user_interface
• Algebraic Effects for the Rest of Us. https://overreacted.io/zh-hans/algebraic-effects-for-the-rest-of-us/
• What does algebraic effects mean in FP? https://stackoverflow.com/questions/49626714/what-does-algebraic-effects-mean-in-fp
• A Distributed Systems Reading List. https://dancres.github.io/Pages/
• Tutte Embedding for Parameterization. https://isaacguan.github.io/2018/03/19/Tutte-Embedding-for-Parameterization/
• Commentary on the Sixth Edition UNIX Operating System. http://www.lemis.com/grog/Documentation/Lions/index.php
• Personal knowledge management beyond versioning. https://dl.acm.org/doi/10.1145/2362456.2362492
• The Plain Text Life: Note Taking, Writing and Life Organization Using Plain Text Files. http://www.markwk.com/plain-text-life.html
• Post-Evernote: How to Migrate Your Evernote Notes, Images and Tags Into Plain Text Markdown. http://www.markwk.com/migrate-evernote-plaintext.html
• Five Levels of Error Handling in Both Python and JavaScript. https://dev.to/jesterxl/five-levels-of-error-handling-in-both-python-and-javascript-13ok

Read these articles:

• The UNIXHATERS Handbook. http://web.mit.edu/~simsong/www/ugh.pdf
• Why are video games graphics (still) a challenge? Productionizing rendering algorithms. https://bartwronski.com/2020/12/27/why-are-video-games-graphics-still-a-challenge-productionizing-rendering-algorithms/
• BPF and Go: Modern forms of introspection in Linux. https://medium.com/bumble-tech/bpf-and-go-modern-forms-of-introspection-in-linux-6b9802682223
• Systems design explains the world: volume 1. https://apenwarr.ca/log/20201227
• Error handling guidelines for Go. https://jayconrod.com/posts/116/error-handling-guidelines-for-go
• The Missing Semester of Your CS Education. https://missing.csail.mit.edu/
• Build your own React. https://pomb.us/build-your-own-react/

Fan-In multiplexes multiple input channels onto one output channel.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

func Funnel(sources ...<-chan int) <-chan int {
    dest := make(chan int)                  // The shared output channel

    var wg sync.WaitGroup                   // Used to automatically close dest
                                            // when all sources are closed

    wg.Add(len(sources))                    // Increment the sync.WaitGroup

    for _, ch := range sources {            // Start a goroutine for each source
        go func(c <-chan int) {
            defer wg.Done()                 // Notify WaitGroup when c closes

            for n := range c {
                dest <- n
            }
        }(ch)
    }

    go func() {                             // Start a goroutine to close dest
        wg.Wait()                           // after all sources close
        close(dest)
    }()

    return dest
}

Fan-Out evenly distributes messages from an input channel to multiple output channels.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

func Split(source <-chan int, n int) []<-chan int {
    dests := make([]<-chan int, 0)          // Create the dests slice

    for i := 0; i < n; i++ {                // Create n destination channels
        ch := make(chan int)
        dests = append(dests, ch)

        go func() {                         // Each channel gets a dedicated
            defer close(ch)                 // goroutine that competes for reads

            for val := range source {
                ch <- val
            }
        }()
    }

    return dests
}

Future provides a placeholder for a value that’s not yet known.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

type Future interface {
    Result() (string, error)
}

type InnerFuture struct {
    once sync.Once
    wg   sync.WaitGroup

    res   string
    err   error
    resCh <-chan string
    errCh <-chan error
}

func (f *InnerFuture) Result() (string, error) {
    f.once.Do(func() {
        f.wg.Add(1)
        defer f.wg.Done()
        f.res = <-f.resCh
        f.err = <-f.errCh
    })

    f.wg.Wait()

    return f.res, f.err
}

func SlowFunction(ctx context.Context) Future {
    resCh := make(chan string)
    errCh := make(chan error)

    go func() {
        select {
        case <-time.After(time.Second * 2):
            resCh <- "I slept for 2 seconds"
            errCh <- nil
        case <-ctx.Done():
            resCh <- ""
            errCh <- ctx.Err()
        }
    }()

    return &InnerFuture{resCh: resCh, errCh: errCh}
}

Sharding splits a large data structure into multiple partitions to localize the effects of read/write locks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

type Shard struct {
    sync.RWMutex                            // Compose from sync.RWMutex
    m map[string]interface{}                // m contains the shard's data
}

type ShardedMap []*Shard                    // ShardedMap is a *Shards slice

func NewShardedMap(nshards int) ShardedMap {
    shards := make([]*Shard, nshards)       // Initialize a *Shards slice

    for i := 0; i < nshards; i++ {
        shard := make(map[string]interface{})
        shards[i] = &Shard{m: shard}
    }

    return shards                           // A ShardedMap IS a *Shards slice!
}

func (m ShardedMap) getShardIndex(key string) int {
    checksum := sha1.Sum([]byte(key))       // Use Sum from "crypto/sha1"
    hash := int(checksum[17])               // Pick a random byte as our hash
    index := hash % len(shards)             // Mod by len(shards) to get index
}

func (m ShardedMap) getShard(key string) *Shard {
    index := m.getShardIndex(key)
    return m[index]
}

func (m ShardedMap) Get(key string) interface{} {
    shard := m.getShard(key)
    shard.RLock()
    defer shard.RUnlock()

    return shard.m[key]
}

func (m ShardedMap) Set(key string, value interface{}) {
    shard := m.getShard(key)
    shard.Lock()
    defer shard.Unlock()

    shard.m[key] = value
}

func (m ShardedMap) Keys() []string {
    keys := make([]string, 0)               // Create an empty keys slice
    wg := sync.WaitGroup{}                  // Create a wait group and add a
    wg.Add(len(m))                          // wait value for each slice
    for _, shard := range m {               // Run a goroutine for each slice
        go func(s *Shard) {
            s.RLock()                       // Establish a read lock on s
            for key, _ := range s.m {       // Get the slice's keys
                keys = append(keys, key)
            }
            s.RUnlock()                     // Release the read lock
            wg.Done()                       // Tell the WaitGroup it's done
        }(shard)
    }
    wg.Wait()                               // Block until all reads are done
    return keys                             // Return combined keys slice
}