The stock `Base.Docs.Binding(m, v)` constructor keeps the local symbol
`v` while normalising the module via `binding_module(m, v)`. For renamed
imports such as `using X: y as z`, this produces a binding to a
non-existent `canonical_owner.z`, so `?` doc lookup fails:
julia> using Base: isvisible as iv
help?> iv
No documentation found.
Binding `Base.iv` does not exist.
Resolve the alias by walking the binding partition chain: for each
explicit by-name import (`PARTITION_KIND_EXPLICIT` /
`PARTITION_KIND_IMPORTED`), follow the partition's restriction to its
underlying `Core.Binding` and read the canonical `(mod, name)` from its
globalref. Iterate until a non-explicit-import partition is reached,
then defer to `binding_module` for any remaining implicit-import
normalisation.
The iterative walk is needed because neither primitive alone suffices
for chained re-exports: `partition_restriction` only walks one hop
(stopping at the intermediate module for a plain re-export of an
`as`-renamed binding), while `binding_module` walks all hops but drops
the name (losing the canonical symbol when an `as`-rename appears
anywhere in the chain).
The first reland attempt rewrote unconditionally on any explicit by-name
import, which broke the docs build for plain chained re-exports such as
`Libdl.DL_LOAD_PATH` (whose docstring lives one hop upstream of the
local import). See
https://buildkite.com/julialang/julia-master/builds/57392#019e4dd3-f9a5-42dc-984d-374451e1a32b
for the original failure.
Re-lands #61869, originally landed as #55119.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
An attempt to reduce garbage debuginfo that codegen needs to deal with:
stop dropping provenance when producing generator methods in lowering,
and generated expressions from `generated_body_to_codeinfo`. This change
appears to let us rule out a case in codegen, but I'll see what CI
thinks. The other changes are a provenance improvement regardless.
`typegroup` had a section in the manual but no keyword docstring, so
`?typegroup` at the REPL returned no documentation. Add a
`kw"typegroup"` entry alongside `struct` and `mutable struct` in
`base/docs/basedocs.jl`.
This pull request was written with the assistance of generative AI
(Claude).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without this change, if ORC materialization fails with an error, we
instead see an assertion failure because of an unchecked Error rather
than seeing the ORC error. Improve the debugging experience by
"successfully" abandoning the allocation, even though we don't support
deallocating code. #61847 correctly prints the
`UnexpectedSymbolDefinitions` error that is the cause of the problem
after this change.
For example, we get this error message instead of a deadlock:
```
nthreads=32
starting workers
JIT session error: Unexpected definitions in module leaf_62-jitted-178: [ julia_leaf_62_0#0, julia_mid_59_0#0 ]
Internal error: Lookup failed: Failed to materialize symbols: { (JuliaOJIT, { julia_topB_52_0#0, jfptr_topB_52_0#0 }) }
Program aborted due to an unhandled Error:
Failed to materialize symbols: { (JuliaOJIT, { julia_topB_52_0#0, jfptr_topB_52_0#0 }) }
```
previously:
```julia
# WRONG. output should be length 5, not 6. the last value here is junk from `similar`
julia> map(&, rand(6) .> 0.5, rand(5) .> 0.5)
6-element BitVector:
0
0
0
1
0
0
julia> dest = falses(64);
julia> map!(&, dest, trues(65), trues(64));
# what on earth?? should be 64
julia> count(dest)
1
```
the `and_iteratorsize` change is slightly drive-by but it makes the
implementation nicer (and more parallel to `Array`, so if one changes
they'll stay in sync). incidentally, it actually helps performance for
some `collect(zip(..))` calls too
```julia
julia> using BenchmarkTools
julia> @btime collect(z) setup=z=zip(1:10000, rand(100, 100));
15.875 μs (16 allocations: 437.53 KiB) # master
3.344 μs (3 allocations: 160.06 KiB) # PR
```
codeveloped (mostly tests) with codex 5.5
---------
Co-authored-by: Nathan Zimmerberg <39104088+nhz2@users.noreply.github.com>
I recently noticed that the doc string for `Base.@ccallable` is not
included in the docs. Given how often it is used with PackageCompiler.jl
and JuliaC.jl at this point, I assume it was just an oversight, and not
an intentional exclusion.
This PR updates the semantics of free TypeVars in subtyping to act as
egality-compared atoms (as proposed in
https://github.com/JuliaLang/julia/issues/61242#issuecomment-4008886548).
Previously free typevars
in subtyping were undefined behavior although they generally behaved
universally (unless the same typevar by identity was present in a
UnionAll). This fixes#61242 by making sure that `Vector !=
Vector.body`.
Written by GPT 5.5.
---------
Co-authored-by: Keno Fischer <Keno@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
The stock `Base.Docs.Binding(m, v)` constructor keeps the local symbol
`v` while normalising the module via `binding_module(m, v)`. For renamed
imports such as `using X: y as z`, this produces a binding to a
non-existent `canonical_owner.z`, so `?` doc lookup fails:
julia> using Base: isvisible as iv
help?> iv
No documentation found.
Binding `Base.iv` does not exist.
Resolve the alias via the binding partition system: when the local
binding's partition kind is `PARTITION_KIND_EXPLICIT` or
`PARTITION_KIND_IMPORTED` — the only kinds whose surface syntax (`using
X: y as z` / `import X: y as z`) admits an `as`-rename — the partition's
restriction is the underlying `Core.Binding` whose globalref carries the
canonical `(mod, name)` pair. Use that pair instead.
Re-lands #55119 with the binding partition API after the original
`jl_binding_owner`-based fix became stale (that C function no longer
exists on master).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pop quiz: Which of these make `T` diagonal:
```
Tuple{S, T} where S<:T where T
Tuple{S, T} where S<:Tuple{T} where T
Tuple{S, T} where S<:Tuple{T,T} where T
```
The answer right now is 2 and 3, but I would argue that 2 is a bit of an
accident of implementation, because `Tuple` turns back on the variable
occurrence counting that the typevar consitency check tried to turn off
(but we depend on `3` and it clearly "looks" diagonal).
This PR changes the semantics so that 2 is no longer considered
diagonal. Instead, the diagonal rule conceptually applies to `Tuple`s:
The diagonal rule applies whenever any `Tuple` sees 2+ active typevars
in covariant position (and the typevar was not statically found in
invariant position by the outer UnionAll - same as before).
For a more extensive set of examples of cases that change behaviors, see
the test suite, but of note none of our existing tests depended on this
(other than a test_broken that now passes).
A particular motivation for this change is intersection, since
intersection needs to extensively reason about whether or not it is
possible for a variable to become diagonal (no intersection changes are
made in this PR, because intersection is not currently correct with
respect to various diagonal cases - however fixing that requires
defining what correctness means).
Idea by me. Tests by GPT-5.5 Pro. Implemented by Claude Opus 4.7.
In the recent discussion about `main`, `__main__` was proposed. One
reason in favor of this is that double underscore, or dunder, names are
reserved names in Julia. This however is not documented.
This pull request adds a paragraph reserving double underscore names for
Julia.
They are added to Base -> Essentials -> Keywords where other reserved
names are documented.
https://docs.julialang.org/en/v1.11-dev/base/base/#Keywords
cc: @StefanKarpinski
---------
Co-authored-by: 3f6a <126961207+3f6a@users.noreply.github.com>
## Summary
`jl_reflect_clone_targets` (exposed as `Base.current_image_targets()`)
was re-resolving `jl_options.cpu_target` from scratch via
`jl_get_llvm_clone_targets`, returning the originally-configured target
spec list rather than the JIT target that `match_pkgimg_target` actually
compares pkgimage clones against. The JIT target (`jit_targets.front()`)
differs from the configured one because it is:
- intersected with the loaded sysimage's clone features,
- clamped by max vector-register width (e.g., AVX-512 features are
stripped when the matched sysimage clone caps at AVX2).
As a result, `JULIA_DEBUG=loading` printed identical "Image Targets" and
"Current Targets" feature strings on pkgimage rejection — hiding the
real mismatch and making the rejection look spurious. See #61621 for two
examples.
This change makes `jl_reflect_clone_targets` serialize `jit_targets`
directly, with a fallback to the prior behavior in case it has not yet
been populated.
Fixes#61621
Pull request was written with the assistance of generative AI (Claude).
## Test plan
- [ ] Trigger a pkgimage rejection (e.g. load a pkgimage built against a
different sysimage clone) with `JULIA_DEBUG=loading` and confirm
"Current Targets" now shows the post-match JIT feature set rather than
the unmatched cpu_target spec.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fix is fully claude one shot. I was also able to glean a small
reproducible test, but it was not deterministic and I don't have
confidence it would work on all machines, so I omitted it.
------------------------------------------------------------------------------------------
This `_Atomic(int8_t)` field was added in #57421 but never initialized
in `jl_new_module__`. `jl_gc_alloc` does not zero the returned memory,
so each fresh module started with whatever garbage was in the pool slot.
The field is read by `all_usings_unchanged_implicit` in
`src/staticdata.c` during pkgimage loading: a non-zero garbage value
spuriously forces binding-partition revalidation for downstream modules.
The fix matches the existing zero-init of the adjacent `has_reexports`
field that was added in #59859.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
The comment suggests that the `correct` variable holds the precise
intersection, but this is false. The precise intersection is (as far as
I can tell) not representable. While we're here, add a related
unsoundness that Claude found while looking at this. The common thread
is that these cases involve what Claude calls `F-bounded witnesses`,
i.e. types of the form `Ptr{T} where T<:Ptr`. I haven't read enough of
the literature on F-bounded polymorphism to know if this is a good name
or not, but for want of a better name, let's go with it.
Image objects are already never freed and are ~somewhat rarely mutated,
making them candidates for "pretenuring".
Load image objects as permanently marked (`GC_OLD_MARKED`) so
`gc_try_setmark_tag` returns 0 immediately and the mark phase never
enters the image subgraph. Maintain any mutations as a dedicated
`image_remset` which effectively roots any "new" referents that are
referred to by the image subgraph.
This significantly speeds up (full) GC pause times when dominated by
immutable objects in the sysimage / pkgimage heaps.
This can likely also be generalized slightly to a "permalloc / pretenure"
operation that applies to objects not necessarily in images. I'm not sure
whether it'd be possible to "unfreeze" those objects once they are
promoted to permalloc'd / pretenured though.
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This change adds a basic C API for reading `jl_debuginfo_t`, both in its
current form and with byte-precise provenance to be generated by
JuliaLowering.
```
(debuginfo, pc) -> (firstbyte, lastbyte)
(debuginfo, byte) -> (line, column)
(debuginfo, pc) -> (firstline, firstcolumn)
debuginfo -> first_line_of_entire_codeinfo
```
Note that the byte-precise provenance is not yet produced (and I've
marked those branches unreachable). I'm making this change first so I
can use them to try and resolve issues in #61699 and #53925. (Namely,
I'm going to try and compute inlined frames from julia debuginfo
rather than DWARF.)
## Implementation of byte-precise info storage
The API shouldn't depend on how we do this, but I'll describe it here
for reference.
The simplicity of our current `Core.DebugInfo` (ref. #52415) is thanks
to simple line numbers looking the same as indices into the next
debuginfo, so they can be stored in the same way. Unfortunately,
byte-precise information breaks this:
from an index, we want to be able to know two byte positions, what lines
they fall on, and how far from the beginning of those lines they are.
I intend to store this info by putting a string in `linetable` like we
do for `di.codelocs`. The compression format is described in julia.h, but
only a couple points are relevant here:
- Like `di.codelocs` today, I made a reasonable attempt to compress it
(so we don't blow up the sysimg) without trying to be too clever.
- Position lookup for a single IR statement can still be done without
decompressing everything.
I've already implemented the compression/decompression code (in
JuliaLowering), but it's not part of this PR. This format might be torn
out when we start trying to store AST provenance.
---------
Co-authored-by: Cody Tapscott <topolarity@tapscott.me>
I pointed Claude at [perf.julialang.org](https://perf.julialang.org/)
and asked it to find opportunities to improve unusually slow benchmarks.
See
[JuliaCI/julia-ci-timing@main/analysis](https://github.com/JuliaCI/julia-ci-timing/tree/main/analysis)
for some of the scripts it used.
This is what it came up with:
----
Two small, BaseBenchmarks-driven perf fixes that share a theme — when
the
storage of an array already has the layout you want, dispatch to the
bulk
path on a `reinterpret` view instead of going through the per-element
fallback.
### `NTuple{N,E}(::Union{Array,Memory})` for `N >= 32`
The default `_totuple(::Type{All32{E,N}}, itr)` path collects into an
intermediate `Vector` and splats. The `All32` cap exists so we don't
specialize tuple construction on every `N`; this PR keeps that property
(no per-`N` codegen) by adding a single fast path for `Array`/`Memory`
inputs whose `isbits` element type matches the requested tuple element
type. In that case the storage is layout-identical to the tuple, so a
single `reinterpret` view + scalar load returns the tuple in O(1) (plus
a length check).
The fast path lives in `reinterpretarray.jl` because `reinterpret` on
arrays isn't yet defined when `tuple.jl` is loaded. Over-long inputs are
silently truncated to match the iterator-based fallback. Non-matching
eltypes / non-isbits eltypes fall through to the existing path
unchanged.
```
master this PR
NTuple{40,Float64}(::Vector{Float64}) 1.36 μs 7.3 ns
NTuple{150,Float64}(::Vector{Float64}) 4.92 μs 17.0 ns
```
(0 allocations on the fast path, vs. `N+~3` allocations and `~33 N` B
on master.)
### Bulk `rand!(::Array{Complex{T}})` for `T <: HWReal`
`rand!(::AbstractRNG, ::Array{Complex{T}})` previously fell through to
the generic `AbstractArray` path — two scalar `rand` calls per element.
A packed `Array{Complex{T}}` has the same memory layout as a length-`2N`
`Array{T}`, so reinterpreting and dispatching to the bulk `rand!` for
`T` (SIMD-vectorized for `HWReal` on Xoshiro/TaskLocalRNG, and pointer-
based for MersenneTwister) is correct and substantially faster.
Per @vtjnash's feedback in earlier review, this avoids
`unsafe_wrap`/`unsafe_load` entirely. Instead, the existing bulk
dispatches in `MersenneTwister` and `XoshiroSimd` are widened to also
accept a `NonReshapedReinterpretArray` over a `MutableDenseArrayType`
parent. `pointer`/`unsafe_convert` are already defined for
`ReinterpretArray` and forward to the parent's storage, so the existing
`GC.@preserve A ... pointer(A) ...` machinery keeps working unchanged.
Speedups on a length-1000 `Vector{Complex{T}}`, default RNG:
```
master this PR
Complex{Float16} 3.5 μs 1.5 μs 2.4×
Complex{Float32} 3.4 μs 2.2 μs 1.5×
Complex{Float64} 3.1 μs 3.1 μs ≈
Complex{Int8} 3.1 μs 539 ns 5.7×
Complex{Int16} 3.1 μs 1.1 μs 3.0×
Complex{Int32} 3.1 μs 1.9 μs 1.7×
Complex{Int64} 4.6 μs 4.2 μs 1.1×
Complex{UInt8} 3.1 μs 532 ns 5.9×
Complex{UInt16} 3.1 μs 1.0 μs 2.9×
Complex{UInt32} 3.0 μs 1.9 μs 1.6×
Complex{UInt64} 4.7 μs 4.2 μs 1.1×
```
All variants are 0-allocation. `Complex{Float64}` and the 64-bit integer
variants were already close to the bulk path's throughput on master and
see only a small win.
A reproducibility test was added so equal MT seeds keep producing equal
sequences for `Vector{Complex{T}}`.
---
Disclosure: written with the assistance of generative AI (GitHub
Copilot / Claude).
---------
Co-authored-by: GitHub Copilot <noreply@github.com>
The processor_{x86,arm,fallback}.cpp files that included these headers
were deleted in #61292, which replaced the hand-maintained CPU/feature
tables with the [cpufeatures](https://github.com/JuliaLang/cpufeatures)
library. No remaining code in the tree references features_x86.h,
features_aarch32.h, or features_aarch64.h, nor the JL_FEATURE_DEF macro
they defined.
#61292 listed these headers under Files to delete.
The constructors documentation used `new` with implicit and explicit
type parameters in examples without clearly stating the underlying rule.
This adds a subsection explaining that `new` inherits the type
parameters from the curly braces on the inner constructor name, when
explicit parameters are needed, and what happens when the constructor
name has fewer parameters than the type requires. The existing
explanation in the "Outer-only constructors" section is also updated to
reference this rule.
The rules described here were determined empirically, as the existing
documentation did not specify them. If any details are inaccurate,
corrections are welcome.
---------
Co-authored-by: James Wrigley <JamesWrigley@users.noreply.github.com>
