spec icon indicating copy to clipboard operation
spec copied to clipboard

Write specs for new Ruby 3.1 features and changes

Open eregon opened this issue 3 years ago • 9 comments

ruby/spec already contains some specs for 3.1, but we should aim to cover all new features and important changes. This will improve the test coverage of these features (and maybe discover a few bugs along the way), allow other Ruby implementations to implement the changes faster with more confidence and document clearly the new behavior.

Please take a look at open pull requests before starting on a task to avoid duplicated work.

The new specs should be within a version guard block:

ruby_version_is "3.1" do
  # New specs
end

NOTE: https://rubyreferences.github.io/rubychanges/3.1.html gives more details for many features and changes.

From https://github.com/ruby/ruby/blob/master/doc/NEWS/NEWS-3.1.0.md:

NEWS for Ruby 3.1.0

This document is a list of user-visible feature changes since the 3.0.0 release, except for bug fixes.

Note that each entry is kept to a minimum, see links for details.

Language changes

  • [x] https://github.com/ruby/spec/commit/bcbc45e3c9c8f2d4ca912013d93dfb03a96b45d8 The block argument can now be anonymous if the block will only be passed to another method. [Feature #11256]

    def foo(&)
      bar(&)
    end
    
  • [x] Pin operator now takes an expression. [Feature #17411]

    Prime.each_cons(2).lazy.find_all{_1 in [n, ^(n + 2)]}.take(3).to_a
    #=> [[3, 5], [5, 7], [11, 13]]
    
  • [x] Pin operator now supports instance, class, and global variables. [Feature #17724]

    @n = 5
    Prime.each_cons(2).lazy.find{_1 in [n, ^@n]}
    #=> [3, 5]
    
  • [x] One-line pattern matching is no longer experimental.

  • [x] Parentheses can be omitted in one-line pattern matching. [Feature #16182]

    [0, 1] => _, x
    {y: 2} => y:
    x #=> 1
    y #=> 2
    
  • [x] Multiple assignment evaluation order has been made consistent with single assignment evaluation order. With single assignment, Ruby uses a left-to-right evaluation order. With this code:

    foo[0] = bar
    

    The following evaluation order is used:

    1. foo
    2. bar
    3. []= called on the result of foo

    In Ruby before 3.1.0, multiple assignment did not follow this evaluation order. With this code:

    foo[0], bar.baz = a, b
    

    Versions of Ruby before 3.1.0 would evaluate in the following order

    1. a
    2. b
    3. foo
    4. []= called on the result of foo
    5. bar
    6. baz= called on the result of bar

    Starting in Ruby 3.1.0, the evaluation order is now consistent with single assignment, with the left-hand side being evaluated before the right-hand side:

    1. foo
    2. bar
    3. a
    4. b
    5. []= called on the result of foo
    6. baz= called on the result of bar

    [Bug #4443]

  • [x] Values in Hash literals and keyword arguments can be omitted. [Feature #14579]

    For example,

    {x:, y:} is a syntax sugar of {x: x, y: y}. foo(x:, y:) is a syntax sugar of foo(x: x, y: y).

    Constant names, local variable names, and method names are allowed as key names. Note that a reserved word is considered as a local variable or method name even if it's a pseudo variable name such as self.

  • [x] Non main-Ractors can get instance variables (ivars) of classes/modules if ivars refer to shareable objects. [Feature #17592]

  • [x] A command syntax is allowed in endless method definitions, i.e., you can now write def foo = puts "Hello". Note that private def foo = puts "Hello" does not parse. [Feature #17398]

Core classes updates

Note: We're only listing outstanding class updates.

Array

Class

  • [x] Class#subclasses, which returns an array of classes directly inheriting from the receiver, not including singleton classes. [Feature #18273]

      ```ruby
      class A; end
      class B < A; end
      class C < B; end
      class D < A; end
      A.subclasses    #=> [D, B]
      B.subclasses    #=> [C]
      C.subclasses    #=> []
      ```
    

Enumerable

  • [x] Enumerable#compact is added. [Feature #17312]

  • [x] Enumerable#tally now accepts an optional hash to count. [Feature #17744]

  • [x] Enumerable#each_cons and each_slice to return a receiver. [GH-1509]

      ```ruby
      [1, 2, 3].each_cons(2){}
      # 3.0 => nil
      # 3.1 => [1, 2, 3]
    
      [1, 2, 3].each_slice(2){}
      # 3.0 => nil
      # 3.1 => [1, 2, 3]
      ```
    

Enumerator::Lazy

File

  • [x] File.dirname now accepts an optional argument for the level to strip path components. [Feature #12194]

GC

  • [x] "GC.measure_total_time = true" enables the measurement of GC. Measurement can introduce overhead. It is enabled by default. GC.measure_total_time returns the current setting. GC.stat[:time] or GC.stat(:time) returns measured time in milli-seconds. [[Feature #10917]]

  • [x] GC.total_time returns measured time in nano-seconds. [[Feature #10917]]

Integer

Kernel

  • [x] Kernel#load now accepts a module as the second argument, and will load the file using the given module as the top-level module. [Feature #6210]

Marshal

  • [x] Marshal.load now accepts a freeze: true option. All returned objects are frozen except for Class and Module instances. Strings are deduplicated. [Feature #18148]

MatchData

Method / UnboundMethod

  • [x] Method#public?, Method#private?, Method#protected?, UnboundMethod#public?, UnboundMethod#private?, UnboundMethod#protected? have been added. [Feature #11689]

Module

  • [x] Module#prepend now modifies the ancestor chain if the receiver already includes the argument. Module#prepend still does not modify the ancestor chain if the receiver has already prepended the argument. [Bug #17423]

  • [x] Module#private, #public, #protected, and #module_function will now return their arguments. If a single argument is given, it is returned. If no arguments are given, nil is returned. If multiple arguments are given, they are returned as an array. [Feature #12495]

Process

  • [x] Process._fork is added. This is a core method for fork(2). Do not call this method directly; it is called by existing fork methods: Kernel.#fork, Process.fork, and IO.popen("-"). Application monitoring libraries can overwrite this method to hook fork events. [Feature #17795]

Struct

  • [x] Passing only keyword arguments to Struct#initialize is warned. You need to use a Hash literal to set a Hash to a first member. [Feature #16806]

  • [x] StructClass#keyword_init? is added [Feature #18008]

String

  • [x] Update Unicode version to 13.0.0 [Feature #17750] and Emoji version to 13.0 [Feature #18029]

  • [x] String#unpack and String#unpack1 now accept an offset: keyword argument to start the unpacking after an arbitrary number of bytes have been skipped. If offset is outside of the string bounds ArgumentError is raised. [Feature #18254]

Thread

Thread::Backtrace

  • [x] Thread::Backtrace.limit, which returns the value to limit backtrace length set by --backtrace-limit command line option, is added. [Feature #17479]

Thread::Queue

  • [x] Thread::Queue.new now accepts an Enumerable of initial values. [Feature #17327]

Time

  • [x] Time.new now accepts optional in: keyword argument for the timezone, as well as Time.at and Time.now, so that is now you can omit minor arguments to Time.new. [Feature #17485]

      ```ruby
      Time.new(2021, 12, 25, in: "+07:00")
      #=> 2021-12-25 00:00:00 +0700
      ```
    
      At the same time, time component strings are converted to
      integers more strictly now.
    
      ```ruby
      Time.new(2021, 12, 25, "+07:30")
      #=> invalid value for Integer(): "+07:30" (ArgumentError)
      ```
    
      Ruby 3.0 or earlier returned probably unexpected result
      `2021-12-25 07:00:00`, not `2021-12-25 07:30:00` nor
      `2021-12-25 00:00:00 +07:30`.
    
  • [x] Time#strftime supports RFC 3339 UTC for unknown offset local time, -0000, as %-z. [Feature #17544]

TracePoint

  • [x] TracePoint.allow_reentry is added to allow reenter while TracePoint callback. [Feature #15912]

$LOAD_PATH

  • [x] $LOAD_PATH.resolve_feature_path does not raise. [Feature #16043]

Fiber Scheduler

  • [ ] Add support for Addrinfo.getaddrinfo using address_resolve hook. [Feature #17370]

  • [ ] Introduce non-blocking Timeout.timeout using timeout_after hook. [Feature #17470]

  • [ ] Introduce new scheduler hooks io_read and io_write along with a low level IO::Buffer for zero-copy read/write. [Feature #18020]

  • [ ] IO hooks io_wait, io_read, io_write, receive the original IO object where possible. [Bug #18003]

  • [ ] Make Monitor fiber-safe. [Bug #17827]

  • Replace copy coroutine with pthread implementation. [Feature #18015]

Refinement

  • [x] New class which represents a module created by Module#refine. include and prepend are deprecated, and import_methods is added instead. [Bug #17429]

Stdlib updates

  • The following default gem are updated.
    • RubyGems 3.3.3
    • base64 0.1.1
    • benchmark 0.2.0
    • bigdecimal 3.1.1
    • bundler 2.3.3
    • cgi 0.3.1
    • csv 3.2.2
    • date 3.2.2
    • did_you_mean 1.6.1
    • digest 3.1.0
    • drb 2.1.0
    • erb 2.2.3
    • error_highlight 0.3.0
    • etc 1.3.0
    • fcntl 1.0.1
    • fiddle 1.1.0
    • fileutils 1.6.0
    • find 0.1.1
    • io-console 0.5.10
    • io-wait 0.2.1
    • ipaddr 1.2.3
    • irb 1.4.1
    • json 2.6.1
    • logger 1.5.0
    • net-http 0.2.0
    • net-protocol 0.1.2
    • nkf 0.1.1
    • open-uri 0.2.0
    • openssl 3.0.0
    • optparse 0.2.0
    • ostruct 0.5.2
    • pathname 0.2.0
    • pp 0.3.0
    • prettyprint 0.1.1
    • psych 4.0.3
    • racc 1.6.0
    • rdoc 6.4.0
    • readline 0.0.3
    • readline-ext 0.1.4
    • reline 0.3.0
    • resolv 0.2.1
    • rinda 0.1.1
    • ruby2_keywords 0.0.5
    • securerandom 0.1.1
    • set 1.0.2
    • stringio 3.0.1
    • strscan 3.0.1
    • tempfile 0.1.2
    • time 0.2.0
    • timeout 0.2.0
    • tmpdir 0.1.2
    • un 0.2.0
    • uri 0.11.0
    • yaml 0.2.0
    • zlib 2.1.1
  • The following bundled gems are updated.
    • minitest 5.15.0
    • power_assert 2.0.1
    • rake 13.0.6
    • test-unit 3.5.3
    • rexml 3.2.5
    • rbs 2.0.0
    • typeprof 0.21.1
  • The following default gems are now bundled gems.
    • net-ftp 0.1.3
    • net-imap 0.2.2
    • net-pop 0.1.1
    • net-smtp 0.3.1
    • matrix 0.4.2
    • prime 0.1.2
    • debug 1.4.0
  • [ ] Coverage measurement now supports suspension. You can use Coverage.suspend to stop the measurement temporarily, and Coverage.resume to restart it. See [Feature #18176] in detail.

  • [ ] Random::Formatter is moved to random/formatter.rb, so that you can use Random#hex, Random#base64, and so on without SecureRandom. [Feature #18190]

Compatibility issues

Note: Excluding feature bug fixes.

  • rb_io_wait_readable, rb_io_wait_writable and rb_wait_for_single_fd are deprecated in favour of rb_io_maybe_wait_readable, rb_io_maybe_wait_writable and rb_io_maybe_wait respectively. rb_thread_wait_fd and rb_thread_fd_writable are deprecated. [Bug #18003]

Stdlib compatibility issues

  • ERB#initialize warns safe_level and later arguments even without -w. [Feature #14256]

  • lib/debug.rb is replaced with debug.gem

  • Kernel#pp in lib/pp.rb uses the width of IO#winsize by default. This means that the output width is automatically changed depending on your terminal size. [Feature #12913]

  • Psych 4.0 changes Psych.load as safe_load by the default. You may need to use Psych 3.3.2 for migrating to this behavior. [Bug #17866]

C API updates

  • Documented. [GH-4815]

  • rb_gc_force_recycle is deprecated and has been changed to a no-op. [Feature #18290]

Implementation improvements

  • Inline cache mechanism is introduced for reading class variables. [Feature #17763]

  • instance_eval and instance_exec now only allocate a singleton class when required, avoiding extra objects and improving performance. [GH-5146]

  • The performance of Struct accessors is improved. [GH-5131]

  • mandatory_only? builtin special form to improve performance on builtin methods. [GH-5112]

  • Experimental feature Variable Width Allocation in the garbage collector. This feature is turned off by default and can be enabled by compiling Ruby with flag USE_RVARGC=1 set. [Feature #18045] [Feature #18239]

Debugger

  • A new debugger debug.gem is bundled. debug.gem is a fast debugger implementation, and it provides many features like remote debugging, colorful REPL, IDE (VSCode) integration, and more. It replaces lib/debug.rb standard library.

  • rdbg command is also installed into bin/ directory to start and control debugging execution.

error_highlight

A built-in gem called error_highlight has been introduced. It shows fine-grained error locations in the backtrace.

Example: title = json[:article][:title]

If json is nil, it shows:

$ ruby test.rb
test.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError)

title = json[:article][:title]
            ^^^^^^^^^^

If json[:article] returns nil, it shows:

$ ruby test.rb
test.rb:2:in `<main>': undefined method `[]' for nil:NilClass (NoMethodError)

title = json[:article][:title]
                      ^^^^^^^^

This feature is enabled by default. You can disable it by using a command-line option --disable-error_highlight. See the repository in detail.

Miscellaneous changes

  • [x] lib/objspace/trace.rb is added, which is a tool for tracing the object allocation. Just by requiring this file, tracing is started immediately. Just by Kernel#p, you can investigate where an object was created. Note that just requiring this file brings a large performance overhead. This is only for debugging purposes. Do not use this in production. [Feature #17762]

  • [x] Now exceptions raised in finalizers will be printed to STDERR, unless $VERBOSE is nil. [Feature #17798]

  • Add ruby -run -e colorize to colorize Ruby code using IRB::Color.colorize_code.

eregon avatar Feb 28 '22 12:02 eregon

@eregon pls update url for news - https://github.com/ruby/ruby/blob/master/doc/NEWS/NEWS-3.1.0.md

makketagg avatar Jul 19 '22 10:07 makketagg

Done, thanks.

eregon avatar Jul 19 '22 10:07 eregon

It seems the spec for Array#intersect? already exists at core/array/intersect_spec.rb.

"One-line pattern matching is no longer experimental." is also taken care of already in language/pattern_matching_spec.rb.

I'll add one for Struct#keyword_init? now.

citizen428 avatar Oct 08 '22 04:10 citizen428

It seems Core classes updates specs below already exist.

  • Class
    • Class#subclasses
      • 3b7273f0b1c63c5aafe7295ab59e8c82bf2a3124
  • Enumerable
    • Enumerable#compact is added
      • https://github.com/ruby/spec/pull/933
    • Enumerable#tally now accepts an optional hash to count
      • 085270e1dae62e2b2418d651b325e05d66a100d7
  • File
    • File.dirname now accepts an optional argument for the level to strip path components
      • 1db24923ad116a36816946eec6a4d87b66c0ca5a
  • GC
    • "GC.measure_total_time = true" enables the measurement of GC.
    • GC.total_time returns measured time in nano-seconds
      • 5185a7a5c83220b97889148f2fe33426bc3e06e9
  • Integer
    • Integer.try_convert is added
      • 366722d29efed390f948ea39d9be5e531bba5483
  • Marshal
    • Marshal.load now accepts a freeze: true option
      • b2013c3c7202ebb314b95e2bc0795139afeaf197
  • MatchData
    • MatchData#match is added
    • MatchData#match_length is added
      • 4b3ad281baead0a7d45627bbf5ff0410fd452329
  • Module
    • Module#private, #public, #protected, and #module_function will now return their arguments
      • 81935b95603a89fb94e43cf487a01643fee84e5e
  • String
    • Update Unicode version to 13.0.0 and Emoji version to 13.0
      • https://github.com/ruby/spec/blob/5b20199dfe609e3c20beacc00866a05d768e95bd/library/rbconfig/unicode_version_spec.rb#L11-L15
      • https://github.com/ruby/spec/blob/5b20199dfe609e3c20beacc00866a05d768e95bd/library/rbconfig/unicode_emoji_version_spec.rb#L11-L15
    • String#unpack and String#unpack1 now accept an offset: keyword argument
      • d63487be25bc8edb84c195ba04b352f67c477c5d
  • Thread::Queue
    • Thread::Queue.new now accepts an Enumerable of initial values
      • https://github.com/ruby/spec/blob/ff4ad11ded03c4c802e125bcd322a130614ecb92/core/queue/initialize_spec.rb#L10-L12
  • $LOAD_PATH
    • $LOAD_PATH.resolve_feature_path does not raise
      • bdad3e1fb0849dfcf3b8da3d6184a4351d687909

ytjmt avatar Oct 08 '22 13:10 ytjmt

I'll work on this one.

  • [ ] Time.new now accepts optional in: keyword argument for the timezone, as well as Time.at and Time.now, so that is now you can omit minor arguments to Time.new. [Feature #17485]

ohbarye avatar Oct 08 '22 15:10 ohbarye

@ytjmt Thank you, this is really helpful, I checked the done items in the list.

Also I would like to take this occasion to thank @byroot @makketagg @kachick @nobu @bjfish @jeremyevans @duerst @chrisseaton @znz @nirvdrum for contributing those specs, most of them done as soon as the feature got implemented, that is really great and helpful.

eregon avatar Oct 09 '22 10:10 eregon

Parentheses can be omitted in one-line pattern matching. [Feature #16182]

Has already been implemented: https://github.com/ruby/spec/blob/22bb5b2f7d449f65a9442146c0e556e25d8f9426/language/pattern_matching_spec.rb#L1332-L1342

The pinning expression specs have also been implemented: https://github.com/ruby/spec/blob/22bb5b2f7d449f65a9442146c0e556e25d8f9426/language/pattern_matching_spec.rb#L1380-L1402

citizen428 avatar Oct 11 '22 04:10 citizen428

Hi,

May I work on this?

Multiple assignment evaluation order has been made consistent with single assignment evaluation order. With single assignment, Ruby uses a left-to-right evaluation order. With this code: [Bug #4443]

Edit: I had some free time and was curious about this change so kinda created a PR already. Feel free to close it if someone else was supposed to work on it.

dushyantss avatar Mar 05 '23 09:03 dushyantss

@dushyantss Yes of course, since it's not checked, I'll try to review it soon.

eregon avatar Mar 05 '23 14:03 eregon