Skip to content

Conversation

@wilzbach
Copy link
Contributor

@wilzbach wilzbach commented Jan 8, 2018

Turn the complex deprecation on by default.
Follow-up to #7081

This might need some work, depending on what Jenkins tells us.

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @wilzbach!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

@wilzbach wilzbach requested a review from WalterBright as a code owner January 8, 2018 15:15
@wilzbach wilzbach added the Review:WIP Work In Progress - not ready for review or pulling label Jan 8, 2018
@wilzbach
Copy link
Contributor Author

wilzbach commented Jan 8, 2018

Okay after looking a bit into this (see dlang/phobos#6014) I found the following problems:

  • How can typeid(cfloat) by called without triggering a deprecation warning?
  • This already triggers a deprecation warning:
deprecated foo(creal r) {}

So usage of creal etc. needs to be allowed in deprecated (-> https://issues.dlang.org/show_bug.cgi?id=18212)

  • there are quite a bunch of deprecation messages in druntime to to all typeinfo, it's spamy but shouldn't be an issue

  • std.conv, std.format and std.traits will be tricky, e.g.

std/conv.d(205): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(205): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(456): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(570): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(586): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(637): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(686): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(759): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(880): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1024): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1400): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1493): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1575): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1818): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1897): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1935): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead
std/conv.d(1400): Deprecation: use of complex type cdouble is deprecated, use std.complex.Complex!(double) instead

For example:

@ibuclaw
Copy link
Member

ibuclaw commented Jan 8, 2018

Could help by removing all tests using complex and imaginary types?

@wilzbach
Copy link
Contributor Author

wilzbach commented Jan 8, 2018

Could help by removing all tests using complex and imaginary types?

Do you mean for Phobos? Yep, that's one idea, but I guess improving this deprecation check to check whether's in a deprecated scope would be better, because we can ensure that no user code is broken during the deprecation phase.

@ibuclaw
Copy link
Member

ibuclaw commented Jan 8, 2018

Yeah, should be simple. Pass the sc parameter to the function.

Then do a cheeky copy-pasto:

dmd/src/dmd/dsymbol.d

Lines 312 to 326 in c22cba2

// Don't complain if we're inside a deprecated symbol's scope
for (Dsymbol sp = sc.parent; sp; sp = sp.parent)
{
if (sp.isDeprecated())
return false;
}
for (Scope* sc2 = sc; sc2; sc2 = sc2.enclosing)
{
if (sc2.scopesym && sc2.scopesym.isDeprecated())
return false;
// If inside a StorageClassDeclaration that is deprecated
if (sc2.stc & STC.deprecated_)
return false;
}

src/dmd/mars.d Outdated
params.bug10378 = true;
break;
case 14488:
printf("-transition=14488 is no longer needed.\n");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use global.stdmsg?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have seen printf being used here a couple of times, but sure global.stdmsg looks cleaner.

@andralex
Copy link
Member

@ibuclaw ok, ok, that's good stuff. Question for @ibuclaw and @9il: is that likely to cause big troubles? I suspect not, because inlining should eliminate the need for passing individual complex values to small functions. Right?

@ibuclaw what's the calling convention for C's _Complex?

@ibuclaw
Copy link
Member

ibuclaw commented Jan 20, 2018

I suspect not, because inlining should eliminate the need for passing individual complex values to small functions. Right?

Its just the cost of copying to and from xmm registers and the stack whenever you use or modify the struct type. I see no inlining problems of all the trivial operators, particularly if the stuct type is templated.

@ibuclaw
Copy link
Member

ibuclaw commented Jan 20, 2018

@ibuclaw what's the calling convention for C's _Complex?

You mean, how is it passed? And on which target?

@andralex
Copy link
Member

@ibuclaw I wonder if _Complex would be compatible with a struct layout.

@ibuclaw
Copy link
Member

ibuclaw commented Jan 20, 2018

So, Complex!double s = *cast(Complex!double*)&c?

The layout is the same, they just live in different places (depending on target).

@ibuclaw
Copy link
Member

ibuclaw commented Jan 20, 2018

Just for comparison.

LDC's functions were taken from @9il 's post, and were compiled using the following flags.

-O -release -linkonce-templates -betterC -mcpu=native

GDC's functions were taken from explore.dgnu.org and compiled using the following flags.

-O2 -march=native

Native Multiplication.

LDC:

cfloat example.nativeMul(cfloat, cfloat):
  vmovq rax, xmm1
  vmovd xmm1, eax
  shr rax, 32
  vmovd xmm2, eax
  vmovq rax, xmm0
  vmovd xmm0, eax
  shr rax, 32
  vmovd xmm3, eax
  vmulss xmm4, xmm0, xmm2
  vmulss xmm2, xmm3, xmm2
  vfmsub213ss xmm0, xmm1, xmm2
  vfmadd231ss xmm4, xmm3, xmm1
  vmovss dword ptr [rsp - 8], xmm0
  vmovss dword ptr [rsp - 4], xmm4
  vmovsd xmm0, qword ptr [rsp - 8]
  ret

GDC:

cfloat example.nativeMul(cfloat, cfloat):
  vmovq QWORD PTR -24[rsp], xmm1
  vmovss xmm3, DWORD PTR -20[rsp]
  vmovss xmm2, DWORD PTR -24[rsp]
  vmovq QWORD PTR -16[rsp], xmm0
  vmovss xmm1, DWORD PTR -12[rsp]
  vmovss xmm0, DWORD PTR -16[rsp]
  jmp __mulsc3@PLT

std.complex Multiplication.

LDC:

std.complex.Complex!(float).Complex example.stdMul(std.complex.Complex!(float).Complex, std.complex.Complex!(float).Complex):
  vmovq rax, xmm1
  vmovd xmm1, eax
  shr rax, 32
  vmovd xmm2, eax
  vmovq rax, xmm0
  vmovd xmm0, eax
  vmulss xmm3, xmm1, xmm0
  shr rax, 32
  vmovd xmm4, eax
  vmulss xmm5, xmm2, xmm4
  vsubss xmm3, xmm3, xmm5
  vmulss xmm0, xmm2, xmm0
  vmulss xmm1, xmm1, xmm4
  vaddss xmm0, xmm1, xmm0
  vmovss dword ptr [rsp - 4], xmm0
  vmovss dword ptr [rsp - 8], xmm3
  vmovsd xmm0, qword ptr [rsp - 8]
  ret

GDC:

std.complex.Complex!(float).Complex example.stdMul(std.complex.Complex!(float).Complex, std.complex.Complex!(float).Complex):
  vmovq rdx, xmm1
  vmovq rcx, xmm0
  vmovdqa xmm3, xmm1
  shr rdx, 32
  shr rcx, 32
  vmovdqa xmm1, xmm0
  vmovd xmm2, edx
  vmovd xmm4, ecx
  vmulss xmm0, xmm4, xmm2
  vfmsub231ss xmm0, xmm3, xmm1
  vmulss xmm1, xmm1, xmm2
  vmovd eax, xmm0
  vfmadd231ss xmm1, xmm3, xmm4
  mov eax, eax
  vmovq rdx, xmm1
  sal rdx, 32
  or rax, rdx
  vmovq xmm0, rax
  ret

Native Divide.

LDC:

cfloat example.nativeDiv(cfloat, cfloat):
  vmovq rax, xmm1
  vmovd xmm1, eax
  shr rax, 32
  vmovd xmm2, eax
  vmovq rax, xmm0
  vmovd xmm0, eax
  shr rax, 32
  vmovd xmm3, eax
  vmulss xmm4, xmm0, xmm1
  vfmadd231ss xmm4, xmm3, xmm2
  vmulss xmm1, xmm3, xmm1
  vfmsub231ss xmm1, xmm0, xmm2
  vmulss xmm0, xmm0, xmm0
  vfmadd231ss xmm0, xmm3, xmm3
  vmovss xmm2, dword ptr [rip + .LCPI2_0]
  vdivss xmm0, xmm2, xmm0
  vmulss xmm2, xmm4, xmm0
  vmulss xmm0, xmm1, xmm0
  vmovss dword ptr [rsp - 8], xmm2
  vmovss dword ptr [rsp - 4], xmm0
  vmovsd xmm0, qword ptr [rsp - 8]
  ret

GDC:

cfloat example.nativeDiv(cfloat, cfloat):
  vmovq QWORD PTR -24[rsp], xmm1
  vmovss xmm3, DWORD PTR -20[rsp]
  vmovss xmm2, DWORD PTR -24[rsp]
  vmovq QWORD PTR -16[rsp], xmm0
  vmovss xmm1, DWORD PTR -12[rsp]
  vmovss xmm0, DWORD PTR -16[rsp]
  jmp __divsc3@PLT

std.complex Divide.

LDC:

std.complex.Complex!(float).Complex example.stdDiv(std.complex.Complex!(float).Complex, std.complex.Complex!(float).Complex):
  push rbx
  sub rsp, 32
  vmovq rax, xmm1
  vmovd xmm1, eax
  vmovd dword ptr [rsp + 12], xmm1
  mov dword ptr [rsp + 16], eax
  shr rax, 32
  vmovd xmm1, eax
  vmovd dword ptr [rsp + 8], xmm1
  mov dword ptr [rsp + 20], eax
  vmovq rbx, xmm0
  vmovd xmm0, ebx
  vmovd dword ptr [rsp + 4], xmm0
  call pure nothrow @nogc @safe float std.math.fabs(float)@PLT
  vmovd dword ptr [rsp + 28], xmm0
  shr rbx, 32
  vmovd xmm0, ebx
  vmovd dword ptr [rsp], xmm0
  call pure nothrow @nogc @safe float std.math.fabs(float)@PLT
  vmovss xmm1, dword ptr [rsp + 28]
  vucomiss xmm0, xmm1
  jbe .LBB3_2
  vmovss xmm0, dword ptr [rsp + 4]
  vmovss xmm1, dword ptr [rsp]
  vdivss xmm2, xmm0, xmm1
  vmulss xmm0, xmm2, xmm0
  vaddss xmm0, xmm0, xmm1
  vmovss xmm3, dword ptr [rsp + 12]
  vmulss xmm1, xmm2, xmm3
  vmovss xmm4, dword ptr [rsp + 8]
  vaddss xmm1, xmm1, xmm4
  vdivss xmm1, xmm1, xmm0
  vmulss xmm4, xmm2, xmm4
  jmp .LBB3_3
.LBB3_2:
  vmovss xmm1, dword ptr [rsp + 4]
  vmovss xmm0, dword ptr [rsp]
  vdivss xmm2, xmm0, xmm1
  vmulss xmm0, xmm2, xmm0
  vaddss xmm0, xmm0, xmm1
  vmovss xmm4, dword ptr [rsp + 8]
  vmulss xmm1, xmm2, xmm4
  vmovss xmm3, dword ptr [rsp + 12]
  vaddss xmm1, xmm1, xmm3
  vdivss xmm1, xmm1, xmm0
  vmulss xmm3, xmm2, xmm3
.LBB3_3:
  vsubss xmm2, xmm4, xmm3
  vdivss xmm0, xmm2, xmm0
  vmovss dword ptr [rsp + 20], xmm0
  vmovss dword ptr [rsp + 16], xmm1
  vmovsd xmm0, qword ptr [rsp + 16]
  add rsp, 32
  pop rbx
  ret

GDC:

std.complex.Complex!(float).Complex example.stdDiv(std.complex.Complex!(float).Complex, std.complex.Complex!(float).Complex):
  vmovq rax, xmm1
  sub rsp, 40
  vmovdqa xmm4, xmm1
  shr rax, 32
  vmovd DWORD PTR 16[rsp], xmm0
  mov DWORD PTR 12[rsp], eax
  vmovq rax, xmm0
  vmovaps xmm0, xmm4
  shr rax, 32
  vmovss DWORD PTR 24[rsp], xmm4
  mov DWORD PTR 28[rsp], eax
  call float std.math.fabs(float)@PLT
  vmovss DWORD PTR 20[rsp], xmm0
  vmovss xmm0, DWORD PTR 12[rsp]
  call float std.math.fabs(float)@PLT
  vucomiss xmm0, DWORD PTR 20[rsp]
  vmovss xmm1, DWORD PTR 28[rsp]
  ja .L67
  vmovss xmm7, DWORD PTR 12[rsp]
  vmovss xmm5, DWORD PTR 24[rsp]
  vmovss xmm6, DWORD PTR 16[rsp]
  vdivss xmm3, xmm7, xmm5
  vmovaps xmm0, xmm6
  vfmadd132ss xmm7, xmm5, xmm3
  vfmadd231ss xmm0, xmm1, xmm3
  vfnmadd231ss xmm1, xmm3, xmm6
  vdivss xmm0, xmm0, xmm7
  vdivss xmm1, xmm1, xmm7
.L63:
  vmovd edx, xmm0
  vmovd eax, xmm1
  add rsp, 40
  sal rax, 32
  mov edx, edx
  or rdx, rax
  vmovq xmm0, rdx
  ret
.L67:
  vmovss xmm4, DWORD PTR 24[rsp]
  vmovss xmm5, DWORD PTR 12[rsp]
  vmovss xmm6, DWORD PTR 16[rsp]
  vdivss xmm2, xmm4, xmm5
  vmovaps xmm0, xmm6
  vfmadd132ss xmm4, xmm5, xmm2
  vfmadd132ss xmm0, xmm1, xmm2
  vfmsub132ss xmm1, xmm6, xmm2
  vdivss xmm0, xmm0, xmm4
  vdivss xmm1, xmm1, xmm4
  jmp .L63

@ibuclaw
Copy link
Member

ibuclaw commented Jan 20, 2018

Notable observations / tl;dr.

  1. LDC native complex is inline.
  2. GDC native complex uses libgcc optimized functions.
  3. GDC std.complex is actually comparable - if not identical - to LDC native. Hurrah!
  4. LDC std.complex sadly is leaving us wanting (this is @9il's concern).
  5. std.complex operator divide could benefit from having less branching, or...
  6. It might be better to use core.math.fabs directly to explicitly use the intrinsic.

@9il
Copy link
Member

9il commented Jan 21, 2018

I can not discus this more. Sorry. My arguments still valid.

@ibuclaw
Copy link
Member

ibuclaw commented Jan 21, 2018

I can not discus this more. Sorry. My arguments still valid.

In what sense though? Just to paraphrase your argument into a digestible format (I hope that I stay true to the meaning in doing so):

  1. D compilers are not able to optimize simple functions in std.complex such as a * b because [reasons/limitations] in inliner.
  2. Even if D compilers can optimize well, it still produces bad code in more complex cases because [reasons/limitations] in optimizer.
  3. We need to add new attributes and semantics to the language frontend in order to overcome these problems in the backend, so that std.complex can produce equivalent code to native complex.

Whereas from what I can see based on your examples, points 1 and 2 are simply not true (see codegen of GDC). It is not outside the capability of a D compiler to generate the most optimal code using std.complex.

And so rather than point 3, it looks like only minor improvements to std.complex to encourage better codegen is needed in some places of phobos. I can comb through this later.

@leandro-lucarella-sociomantic
Copy link
Contributor

However poorly explained, there seems to be evidence removing complex numbers has non-obvious consequences related to efficiency and C compatibility. These are important and deserve further investigation. Just deprecating and then removing complex numbers and letting users deal with it should be preceded by a careful investigation.

@leandro-lucarella-sociomantic, what impact would this have on you?

I'm not sure what do you mean by that, but what I would agree that this should not be deprecated until there is some certainty that the alternative is ready to fully replace the deprecated functionality. There is no point in deprecating something when the alternative is half done, because then you let the user in a wasteland where you either get spurious deprecation messages or you have to start using a half-baked alternative.

@ibuclaw
Copy link
Member

ibuclaw commented Jan 23, 2018

Functionality has been there for years in std.complex. I wouldn't call it half done, but increased use/reliance on it will bring into sharp focus areas that could be improved, either at compile or runtime. Both are not blockers per say, but it will reflect on std.complex positively as far as performance goes.

Only open question I see that has not been discussed is what a hypothetical core.complex would look like and why, and what precisely would be exposed by the compiler @andralex.

@andralex
Copy link
Member

andralex commented Jan 23, 2018

Only open question I see that has not been discussed is what a hypothetical core.complex would look like and why, and what precisely would be exposed by the compiler @andralex.

Intuitively programs should be able to use the complex type, at least the basics (layout and fundamental operations) without needing the standard library.

@wilzbach
Copy link
Contributor Author

Intuitively programs should be able to use the complex type, at least the basics (layout and fundamental operations) without needing the standard library.

We ship the standard library with all releases - it's after all the standard library, so I think you were referring to "usable in -betterC". As mentioned before by @ibuclaw nearly all methods are already templates and thus can be used in -betterC. We can even add a testsuite for -betterC to Phobos, so I don't see the reason for duplicating std.complex to core.complex.

@ibuclaw
Copy link
Member

ibuclaw commented Jan 31, 2018

@kinke you wanted to know some examples, here is one.

@wilzbach
Copy link
Contributor Author

Complex numbers are no bother for those not using them. The last time "complex" appears in the title of a forum post is 2015/03/21. So removal of the feature won't improve their life.
Two notable open bugs on complex numbers, that I think show that there is in fact a little bit of bother using them.

As long as we don't pull the switch here, people will use the built-in complex types and not std.complex or a third-party DUB package (see the recent bug report below).

In contrast, there are other areas of the language that are poorly defined and that do hurt the language size, complexity, and rigor adversely. Any time spent on this by our collaborators is time not spent there, creating a large opportunity cost.

Yes, but the implementation of complex numbers in DMD isn't well tested because not so many people use it and hence people run into bugs:

https://issues.dlang.org/show_bug.cgi?id=18464

I don't think that finding all the places in DMD which need to be special-cased for complex number isn't time well spent either.

Intuitively programs should be able to use the complex type, at least the basics (layout and fundamental operations) without needing the standard library.

Why does it need to be in DRuntime then? Can't we just ensure that the basics work with -betterC? After all, it's already templated...

Also we have made a lot of bad experiences with adding modules to druntime and phobos + the interest of using complex numbers without Phobos seems to be limited to math libraries like mir, so why couldn't this be a DUB package?

@ibuclaw
Copy link
Member

ibuclaw commented Feb 25, 2018

limited to math libraries like mir

Well, if you are a maintainer of a specialization library geared towards math, you probably won't use half of what's available in std anyway.

I still think the only outstanding thing here is ABI compatibility with C. We already have struct __c_long, struct __c_ulong, and struct __c_long_double that are a special library defined type recognized by the compiler for ABI conformity. Why not struct __c_complex_double also?

@p0nce
Copy link
Contributor

p0nce commented Mar 18, 2018

Hello, I'm upgrading to 2.079 and I see builtin complexes continue their descent into deprecation. Many functions in std.math have been deprecated.

Please reconsider, builtin complexes are extremely useful.

I use builtin complex types heavily for signal processing, where they are both more readable and faster.
Made a benchmark to illustrate the sometimes catastrophic performance profile of std.complex "zero-cost" abstraction, that turns out to have a real cost once measured.

BENCHMARK:
https://issues.dlang.org/show_bug.cgi?id=18627

Key takeaway:

We are looking to a guaranteed 2x to 10x performance regression when using complex number arithmetic + DMD

I use builtin complex originally because they are faster (in that I've also found what @9il has found) but also the syntax is an improvement over just a struct. I think this realization will be found by anyone doing heavy number crunching with D.

@ibuclaw
Copy link
Member

ibuclaw commented Mar 20, 2018

I use builtin complex originally because they are faster (in that I've also found what @9il has found) but also the syntax is an improvement over just a struct. I think this realization will be found by anyone doing heavy number crunching with D.

And just for balance, I use std.complex because I found the opposite to be true. :-)

I've already stated here that std.complex is not inherently slow, and the compiler can generate performant code with it.

@p0nce
Copy link
Contributor

p0nce commented Mar 20, 2018

@ibuclaw In your comment you showed that (at least in GDC), builtin complex division and multiplication jump to a library, but there are no measurements that it's faster or slower that way. (EDIT: I see that did measure with the https://issues.dlang.org/show_bug.cgi?id=18627 example, interesting data point)

Builtin complexes are the thing that is being replaced. It's the baseline.
std.complex has to be measured against this baseline, else how do we know we really care about performance. @9il also had a set of more sophisticated remarks.

From the user point of view I don't care if they are kept or not, I only care that the replacement is at least as fast guaranteed. Else it's a performance regression, and when using DMD this performance regression is (currently) guaranteed.

I get it. Most people don't need complex numbers at all. So everyone want them dead, and assume noone is using them (again without asking since 6 years). But some of us are. I've become increasingly admirative of their design since doing a bit of math. It's not like they were added for no reason. Complex numbers are used thoughout numerical programming.

Mostly I trust the inliner, but using LDC there is nothing to trust or not being that builtin complexes are always inlined. Not having to trust > trust.

@wilzbach
Copy link
Contributor Author

DMD this performance regression is (currently) guaranteed.

With this argument, we wouldn't be allowed to use ranges. They are usually 2-3x slower than their naive loop replacement in DMD.

Mostly I trust the inliner, but using LDC there is nothing to trust or not being that builtin complexes are always inlined. Not having to trust > trust.

That's what pragma(inline, true) was made for, no?
It could be easily added to more functions in std.complex.

@p0nce
Copy link
Contributor

p0nce commented Mar 20, 2018

Well, I certainly don't use any range in sensitive numerical processing functions. That's just basic de-risking to me. If you optimize you just can't assume "zero-cost-abstractions" are zero-cost because of their name.

@Geod24
Copy link
Member

Geod24 commented Apr 3, 2021

@ibuclaw has taken this over, so I think we can close this.

@Geod24 Geod24 closed this Apr 3, 2021
@ibuclaw ibuclaw self-assigned this Apr 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Review:Needs Approval Review:Vision Vision Plan https://wiki.dlang.org/Vision/2018H1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants