KernelNewbies
  • Comments
  • Immutable Page
  • Menu
    • Navigation
    • RecentChanges
    • FindPage
    • Local Site Map
    • Help
    • HelpContents
    • HelpOnMoinWikiSyntax
    • Display
    • Attachments
    • Info
    • Raw Text
    • Print View
    • Edit
    • Load
    • Save
  • Login

Kernel Hacking

  • Frontpage

  • Kernel Hacking

  • Kernel Documentation

  • Kernel Glossary

  • FAQ

  • Found a bug?

  • Kernel Changelog

  • Upstream Merge Guide

Projects

  • KernelJanitors

  • KernelMentors

  • KernelProjects

Community

  • Why a community?

  • Regional Kernelnewbies

  • Personal Pages

  • Upcoming Events

References

  • Mailing Lists

  • Related Sites

  • Programming Links

Wiki

  • Recent Changes

  • Site Editors

  • Side Bar

  • Tips for Editors

  • Hosted by WikiWall

Navigation

  • RecentChanges
  • FindPage
  • HelpContents
KernelNewbies:
  • Linux_4.18
Last updated at 2018-11-25 21:40:35

Linux 4.18 has been released on 12 August, 2018.

Summary: This release includes the new "Restartable sequences" system call, which makes easier to write scalable userspace code; support for unprivileged mounts; the beginning of bpfilter project that aims to provide netfilter functionality using BPF; a zero-copy TCP receive API; a new AF_XDP address family for high performance networking; support for the future AMD ATI Vega20; support for the qualcomm Snapdragon 845 SoC; and many new drivers and other improvements.

Contents

  1. Prominent features
    1. Restartable sequences for easier lockless concurrency in user space
    2. Support for unprivileged mounts
    3. bpfilter, BPF based networking filtering
    4. Zero-copy TCP receive API
    5. AF_XDP for high performance networking
    6. Support for the future AMD ATI Vega20
    7. Qualcomm Snapdragon 845 support
  2. Core (various)
  3. File systems
  4. Memory management
  5. Block layer
  6. Tracing and perf
  7. Virtualization
  8. Cryptography
  9. Security
  10. Networking
  11. Architectures
  12. Drivers
    1. Graphics
    2. Storage
    3. Drivers in the Staging area
    4. Networking
    5. Audio
    6. Tablets, touch screens, keyboards, mouses
    7. TV tuners, webcams, video capturers
    8. Universal Serial Bus
    9. Serial Peripheral Interface (SPI)
    10. ACPI, EFI, cpufreq, thermal, Power Management
    11. Real Time Clock (RTC)
    12. Voltage, current regulators, power capping, power supply
    13. Pin Controllers (pinctrl)
    14. Multi Media Card (MMC)
    15. Memory Technology Devices (MTD)
    16. Industrial I/O (iio)
    17. Pulse-Width Modulation (PWM)
    18. Inter-Integrated Circuit (I2C)
    19. Hardware monitoring (hwmon)
    20. Leds
    21. Cryptography hardware acceleration
    22. PCI
    23. Clock
    24. Various
  13. List of merges
  14. Other news sites

1. Prominent features

1.1. Restartable sequences for easier lockless concurrency in user space

Synchronizing per-cpu operations from user-space is hard, because processes are liable to be pre-empted or change cpus at any time. Adding safety to the user-space code (atomic operations) is expensive, and disabling preemption in user space is not an option.

Restartables sequences allow user-space to perform update operations on per-cpu data without requiring heavy-weight atomic operations. Each thread is allowed to register one userspace memory area that should be run by the same CPU; if the thread is preempted the thread can restart the operation. Using rseq with per-cpu memory pools in jemalloc at Facebook improved the production workload response-time 1-2% gain avg. latency, and the P99 overall latency drops by 2-3%.

Recommended LWN article: Restartable sequences

Slides: LPC - PerCpu Atomics.pdf

Code: commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

1.2. Support for unprivileged mounts

In this release, root users inside a private namespace (eg. container) are allowed to mount file systems.

Recommended LWN article: Unprivileged file system mounts

Code: commit, commit, commit, commit, commit, commit, commit

1.3. bpfilter, BPF based networking filtering

The bpfilter project aims to implement netfilter compatible functionality using the BPF machinery. This release adds the first skeleton of kernel code.

Recommended LWN article: BPF comes to firewalls

Code: commit, commit

1.4. Zero-copy TCP receive API

Linux 4.14 added support for zero-copy of data when sending to the network, this release adds a zero-copy API for the receiving side.

Recommended LWN article: Zero-copy TCP receive, A reworked TCP zero-copy receive API

Code: commit, commit, commit, commit, commit, commit

1.5. AF_XDP for high performance networking

Introduce AF_XDP support, a new address family that is optimized for high performance packet processing and zero-copy semantics. Programs using AF_XDP sockets will now receive RX packets without any copies and can also transmit packets without incurring any copies.

Recommended LWN article: Accelerating networking with AF_XDP

Code: commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit zero-copy commit, commit, commit, commit, commit, commit, commit, commit, commit

1.6. Support for the future AMD ATI Vega20

This release includes support for the future Vega 20.

Code: commit, commit, merge

1.7. Qualcomm Snapdragon 845 support

The convergence between vendor Android kernel code and the mainline kernel is improving; as proof of that this release adds support for Qualcomm SDM845, a.k.a Snapdragon 845, an 4+4-core Kryo 385/845 (Cortex-A75/A55 derivative) SoC that's one of the current high-end mobile SoCs. A lot of peripherals are not yet in the DTs but driver support for USB, GPU and other pieces are starting to trickle in. It also adds support for the MTP development board

Code: commit

2. Core (various)

  • (FEATURED) Add support for restartable sequences commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

  • (FEATURED) Asynchronous I/O: add workqueue based fsync offload commit, commit, commit, commit, commit, commit, commit

  • (FEATURED) Add support for unprivileged mounts commit, commit, commit, commit, commit, commit, commit

  • task scheduler: Stagger NUMA balancing scan periods for new threads commit

  • scripts: Add SPDX checker script commit

  • Align the seccomp logging behavior for both audited and unaudited processes. It also emits an audit record, if auditing is enabled, when the kernel.seccomp.actions_logged sysctl is written to so that there's a paper trail when entire actions are quieted commit, commit, commit

  • Add support to collect hardware logs in crash recovery kernel commit, commit, commit

  • signalfd: Add support for SIGSYS commit

  • connector: add parent pid and tgid to coredump and exit events commit

  • Don't schedule threads on pre-empted vCPUs commit, commit

3. File systems

  • BTRFS

    • Add support for the FS_IOC_FSGETXATTR ioctl commit

    • Allow users to enumerate subvolumes with 3 new unprivileged ioctls commit, commit, commit

    • Allow empty subvol= again commit

    • dedupe_file_range ioctl: remove 16MiB restriction commit

    • Allow rmdir(2) to delete an empty subvolume commit

    • send: improved speed when dealing with a large removed directory, measurements show decrease from 2000 minutes to 2 minutes on a directory with 2 million entries commit

    • Pre-commit check of superblock to detect a mysterious in-memory corruption commit

    • sysfs: Add entry which shows if rmdir can work on subvolumes commit

  • XFS

    • Add mount delay debug option, similar to log_recovery_delay, this delay occurs between the VFS superblock being initialised and the xfs_mount being fully initialised commit

    • Support fallocated swapfiles commit

    • Ability to relabel mounted filesystems commit

    • Implement the metadata repair ioctl flag commit

    • Use FUA for pure data O_DSYNC DIO writes commit

  • CIFS

    • Add support for POSIX negotiate context with mount option posix commit, commit, commit, commit, commit

    • Allow disabling less secure legacy dialects with module parameter disable_legacy_dialects commit

  • AFS

    • Enable IPv6 DNS lookups commit

    • Implement network namespacing commit

    • Show all of a server's addresses in /proc/fs/afs/servers commit

  • UDF

    • Add support for UTF-16 characters commit, commit

  • FSCRYPT

    • Add Speck128/256 support commit

  • F2FS

    • Add fsync_mode=nobarrier for non-atomic files commit

4. Memory management

  • Restructuration of struct page, to give more contiguous usable space to users who have allocated a struct page for their own purposes. Recommended LWN article: Willy's memory-management to-do list. commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

  • memcg: introduce the memory.min interface. It works very similarly to memory.low (sharing the same hierarchical behavior), except that it's not disabled if there is no more reclaimable memory in the system commit

  • memcg: allow lowering memory.swap.max below the current usage commit

  • memcg: implement memory.swap.events. Add swap max and fail events so that userland can monitor and respond to running out of swap commit, commit

  • shmem: make stat.st_blksize return huge page size if THP is on commit

  • Reduce the number of TLB shootdowns commit

  • Take better advantage of general huge page clearing optimization commit

5. Block layer

  • Add sysfs entry for fua support commit

  • dm: add writecache target. The writecache target caches writes on persistent memory or SSD. It is intended for databases or other programs that need extremely low commit latency commit

  • loop: Add LOOP_SET_BLOCK_SIZE in compat ioctl commit

  • zram: zram memory tracking. It provides a debugfs /sys/kernel/debug/zram/zram0/block_state to represent each block's state so admin can investigate what memory is cold|incompressible|same page with using pagemap once the pages are swapped out commit, commit, commit, commit

6. Tracing and perf

  • Support perf annotate --group for non-explicit recorded event "groups", showing multiple columns, one for each event, just like when dealing with explicit event groups commit

  • perf buildid-cache: Support --list option option to list all valid cached files commit

  • perf buildid-cache: Support --purge-all option to remove all files from cache commit

  • perf script: Show symbol offsets by default commit

  • perf script: Show virtual addresses instead of offsets commit

  • perf stat: Add --interval-clear option to clear the screen before next interval commit

  • perf stat: Add --table option to display time of each run commit

  • perf stat: Display user and system time commit

  • Add infrastructure to help in writing eBPF C programs to be used with '-e name.c' type events in tools such as 'record' and 'trace', with headers for common constructs and an examples directory that will get populated as we add more such helpers and the 'perf bpf' commit, commit, commit, commit, commit

  • tracing: Allow writing to the trace_markers file initiate triggers defined in tracefs/ftrace/print/trigger file. This will allow of user space to trigger the same type of triggers (including histograms) that the trace events use commit

7. Virtualization

  • Enable virtio_net to act as a standby for a passthru device. The main motivation for this patch is to enable cloud service providers to provide an accelerated datapath to virtio-net enabled VMs in a transparent manner with no/minimal guest userspace changes. This also enables hypervisor controlled live migration to be supported with VMs that have direct attached SR-IOV VF devices commit, commit, commit, commit, commit

  • xen-front: Add support for Xen para-virtualized frontend display driver commit

  • virtio_pci: support enabling VFs commit

  • xen: add IOCTL_PRIVCMD_MMAP_RESOURCE, a new HYPERVISOR_memory_op to support direct priv-mapping of certain guest resources (such as ioreq pages, used by emulators) by a tools domain, rather than having to access such resources via the guest P2M commit

  • xen-front: Add Xen sound para-virtualized frontend driver commit, commit, commit, commit, commit

8. Cryptography

  • zstd: Add zstd support commit

  • aegis: Add the AEGIS family of AEAD algorithms (AEGIS-128, AEGIS-128L, and AEGIS-256) commit

  • morus: Add MORUS-640 AEAD algorithm commit, commit

9. Security

  • apparmor: add support for mapping between the secid and the label, and along with it a basic ability to provide secctx conversion commit

  • apparmor: add the ability to get a task's secid commit

  • EVM: Allow runtime modification of the set of verified xattrs commit

  • integrity: Add an integrity directory in securityfs commit

  • allocation overflow detection helpers commit

  • Remove open-coded multiplication from memory allocation arguments commit, commit

10. Networking

  • TCP

    • (FEATURED) Add zero copy receive commit, commit, commit, commit, commit, commit

    • Implement SACK compression commit, commit, commit, commit, commit, commit

    • Default RACK loss recovery (implements the features correspond to the draft-ietf-tcpm-rack-03 version of the RACK draft) commit, commit, commit, commit, commit, commit, commit

    • Tracking TCP data delivery and ECN stats commit, commit, commit, commit

    • Add the TCP_INQ socket option to TCP. When this socket option is set, recvmsg() relays the number of bytes available on the socket for reading to the application via the TCP_CM_INQ control message. This way applications with many concurrent connections, high variance in receive queue length and tight memory bounds can allocate worst-case buffer size to drain sockets. Knowing the size of receive queue length, applications can optimize how they allocate buffers to read from the socket commit

    • Extend tcp_tw_reuse sysctl to enable loopback only optimization commit

  • UDP:

    • Generic Segmentation Offload support. Segmentation offload reduces cycles/byte for large packets by amortizing the cost of protocol stack traversal. A process can concatenate and submit multiple datagrams to the same destination in one send call by setting socket option SOL_UDP/UDP_SEGMENT with the segment size, or passing an analogous cmsg at send time commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

    • Enable UDP checksum offload for ESP commit

  • BPF

    • (FEATURED) Add skeleton of bpfilter kernel module: it builds experimental bpfilter framework that is aiming to provide netfilter compatible functionality via BPF commit, commit

    • Introduce BTF: BPF Type Format. It is the meta data format which describes the data types of BPF program/map. Hence, it basically focus on the C programming language which the modern BPF is primary using. The first use case is to provide a generic pretty print capability for a BPF map commit, commit, commit, commit, commit, commit, commit, commit, commit

    • Introduce BTF ID - an ID for each loaded BTF program commit, commit, commit, commit, commit, commit

    • Enhancements for multi-function programs commit, commit, commit, commit, commit, commit, commit, commit

    • Introduce seg6local End.BPF action with the corresponding new BPF program type BPF_PROG_TYPE_LWT_SEG6LOCAL commit, commit, commit, commit, commit, commit

    • Hooks for sys_sendmsg similar to existing hooks for sys_bind and sys_connect commit, commit, commit, commit

    • Allows the BPF loader to figure out the btf_key_id and btf_value_id from a map's name by using BPF_ANNOTATE_KV_PAIR commit, commit

    • Allow map helpers access to map values directly commit

    • Hash support for sock commit, commit, commit, commit

    • Introduce bpf subcommand BPF_TASK_FD_QUERY to show which bpf program is attached to which tracepoint/kprobe/uprobe commit, commit, commit, commit, commit

    • Support offload of bpf_event_output() commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

  • (FEATURED) Introducing AF_XDP support, a new address family that is optimized for high performance packet processing and zero-copy semantics. Programs using AF_XDP sockets will now receive RX packets without any copies and can also transmit packets without incurring any copies commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit zero-copy commit, commit, commit, commit, commit, commit, commit, commit, commit

  • Wireless (802.11)

    • Export TXQ parameters and statistics via nl80211 commit, commit

    • Introduces the average ack rssi for data frames commit, commit

    • nl80211: support offloading FILS-SK roaming scenarios commit, commit, commit

    • Add wmm rule attribute to NL80211_CMD_GET_WIPHY dump command commit

  • packet scheduler

    • cls_flower: Support multiple masks per priority commit

  • ipconfig: NTP server support commit, commit, commit, commit, commit, commit, commit, commit

  • IPv4/v6: extends ip route get to match on sport and dport commit, commit

  • IPv6: send netlink notifications for manually configured addresses commit

  • ipvs: Add IPv6 support to IPVS FTP-NAT commit, commit

  • Introduce adaptive TX interrupt moderation to net Dynamic Interrupt Moderation library commit, commit

  • routing: Add address attribute (IFA_RT_PRIORITYto control metric of prefix route. For use cases such as VRR (Virtual Router Redundancy) interface managers want efficient control over the order of prefix routes when multiple interfaces have addresses with overlapping/duplicate subnets commit, commit, commit, commit, commit, commit, commit

  • Add a generic infrastructure to offload TLS crypto to a network devices. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path. Leaving those computationally expensive operations to the NIC commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

  • IPv6 GRE tunnel: Add GRO support, significantly improves performance in case GRO in native interface is disabled commit

  • netfilter

    • Add NAT support for shifted portmap ranges commit

    • ipvs: Implement the Google's Maglev hashing algorithm as a IPVS scheduler. Basically it provides consistent hashing but offers some special features about disruption and load balancing commit, commit, commit

    • nf_tables: add support for native socket matching commit

    • nf_tables: Add audit support to log statement commit

    • nf_tables: add connlimit support commit

    • nf_tables: initial support for extended ACK reporting commit

    • Add support for map lookups to numgen, random and hash expressions commit, commit, commit

  • Extend availability of PHY statistics commit, commit, commit, commit, commit, commit, commit, commit

  • bridge: add support for port isolation. Isolated ports cannot communicate between each other, but they can still communicate with non-isolated ports commit

  • Distributed Switch Architecture: adds PHYLINK support to DSA which is necessary to support more complex PHY and pluggable modules setups commit, commit, commit, commit, commit, commit, commit, commit, commit

  • virtual routing and forwarding: add CRC32c offload to device features commit

  • tun: allow ethtool to change tun link settings commit

  • qrtr: Expose tunneling endpoint to user space commit

  • openvswitch: Support conntrack zone limit commit, commit

  • SCTP: allow unsetting sockopt MAXSEG commit

  • TIPC: Confifuration of MTU for media UDP commit, commit, commit

  • TIPC: introduce ioctl for fetching node identity commit

  • rfkill: Create rfkill-none LED trigger commit

  • Infiniband

    • Add verbs flow counters support. It allows user space applications to monitor real time traffic activity and events of the verbs objects it manages, e.g.: ibv_qp, ibv_wq, ibv_flow commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit, commit

    • MPLS and GRE flow specifications commit, commit, commit, commit, commit, commit, commit

  • SMC

    • Handle ioctls SIOCINQ, SIOCOUTQ, and SIOCOUTQNSD commit

    • Handle sockopts forcing fallback commit

    • Handle sockopt TCP_DEFER_ACCEPT commit

    • sockopts TCP_NODELAY and TCP_CORK commit

    • IPv6 support for smc_diag.c commit

    • Periodic testlink support commit

    • Aadd support for splice() commit

11. Architectures

  • ARM
    • Device Tree Sources

      • SoC: Amlogic Meson 8M2 SoC, a.k.a. S812. A quad Cortex-A9 SoC used in some set top boxes and other products commit

      • SoC: Mediatek MT7623A, which is a flavor of the MT7623 family with other on-chip ethernet options commit

      • (FEATURED) SoC: Qualcomm SDM845, a.k.a Snapdragon 845, an 4+4-core Kryo 385/845 (Cortex-A75/A55 derivative) SoC that's one of the current high-end mobile SoCs. A lot of peripherals are not yet in the DTs but driver support for USB, GPU and other pieces are starting to trickle in. Add also support for the MTP development board