expected results for regtest

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

expected results for regtest

Mark Roberts

Are the expected results for running a full regtest documented anywhere?  I couldn’t find it so I am assuming there should be no failures at all.  Running the release tar file of 3.11.0 does not pass for me on two different versions of Linux (both running on x86_64).

 

 

Fedora 22

== 656 tests, 1 stderr failure, 0 stdout failures, 0 stderrB failures, 1 stdoutB failure, 2 post failures ==

gdbserver_tests/hgtls                    (stdoutB)

memcheck/tests/leak_cpp_interior         (stderr)

massif/tests/new-cpp                     (post)

massif/tests/overloaded-new              (post)

 

 

Ubuntu 15.10

== 656 tests, 6 stderr failures, 0 stdout failures, 0 stderrB failures, 1 stdoutB failure, 2 post failures ==

gdbserver_tests/hgtls                    (stdoutB)

memcheck/tests/leak_cpp_interior         (stderr)

massif/tests/new-cpp                     (post)

massif/tests/overloaded-new              (post)

helgrind/tests/pth_destroy_cond          (stderr)

helgrind/tests/tc09_bad_unlock           (stderr)

helgrind/tests/tc20_verifywrap           (stderr)

drd/tests/recursive_mutex                (stderr)

drd/tests/tc09_bad_unlock                (stderr)

 

 

Is this expected/not expected?  If not expected, how should I proceed?

 

Thank you,

Mark Roberts


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Reply | Threaded
Open this post in threaded view
|

Re: expected results for regtest

John Reiser
> Are the expected results for running a full regtest documented anywhere?  I couldn’t find it so I am assuming there should be no failures at all.  Running the release tar file of 3.11.0 does not pass for me on two different versions of Linux (both running on x86_64).

We need to see the differences in order to make appropriate comments.
Yes, "0-0-0-0-0" is the best possible outcome,
but we might accept some differences depending on what they are.
Some differences might be "style", some might be "evolution".
Others could be real bugs.


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Reply | Threaded
Open this post in threaded view
|

Re: expected results for regtest

John Reiser
In private email, Mark Roberts wrote:
> Thank you for your help.  I have attached the requested files from my
> primary machine - an intel i7-2600 based desktop system running
> Linux(Fedora) 4.3.3-200.fc22.x86_64 with 16G memory.

Thank you for sending detailed information.

I ran "make regtest" from valgrind-svn last night, on Fedora 22 x86_64.
The summary:
== 743 tests, 2 stderr failures, 1 stdout failure, 0 stderrB failures, 1 stdoutB failure, 2 post failures ==
gdbserver_tests/hgtls                    (stdoutB)
memcheck/tests/leak_cpp_interior         (stderr)
massif/tests/new-cpp                     (post)
massif/tests/overloaded-new              (post)
exp-sgcheck/tests/preen_invars           (stdout)
exp-sgcheck/tests/preen_invars           (stderr)

Looking at a couple of my *.diff files [excerpted and snipped aggressively]:

===== memcheck/tests/leak_cpp_interior/leak_cpp_interior.stderr.diff-64bit  (stderr)
-   still reachable: 239 bytes in 8 blocks
+   still reachable: 72,943 bytes in 9 blocks

This is a change that is internal to glibc, perhaps something related to
locale tables.  From memcheck's point of view, the external environment
has changed (using glibc-2.21-13.fc22.x86_64).

===== massif/tests/new-cpp.post.diff   (post)
-  1          4,008            4,008            4,000             8            0
-  2          8,016            8,016            8,000            16            0
-  3         10,024           10,024           10,000            24            0
-  4         12,032           12,032           12,000            32            0
-  5         12,032           12,032           12,000            32            0
-99.73% (12,000B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
-->33.24% (4,000B) 0x........: main (new-cpp.cpp:19)
+  1         72,712           72,712           72,704             8            0
+  2         76,720           76,720           76,704            16            0
+  3         80,728           80,728           80,704            24            0
+  4         82,736           82,736           82,704            32            0
+  5         84,744           84,744           84,704            40            0
+  6         84,744           84,744           84,704            40            0
+99.95% (84,704B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
+->85.79% (72,704B) 0x........: ??? (in /usr/lib64/libstdc++.so.6.0.21)
+| ->85.79% (72,704B) 0x........: call_init.part.0 (dl-init.c:72)
+|   ->85.79% (72,704B) 0x........: _dl_init (dl-init.c:30)
+|     ->85.79% (72,704B) 0x........: ??? (in /usr/lib64/ld-2.21.so)
+|
+->04.72% (4,000B) 0x........: main (new-cpp.cpp:19)

I believe that this change in output from massif has the same underlying cause
as the change above in the output from memcheck.

Thus I believe that there is little cause for worry.

--


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Reply | Threaded
Open this post in threaded view
|

Re: expected results for regtest

iraisr
Hi there,
Thank you for analysing the situation.
Let me clarify things a bit...

2016-04-15 2:56 GMT+02:00 John Reiser <[hidden email]>:
===== memcheck/tests/leak_cpp_interior/leak_cpp_interior.stderr.diff-64bit  (stderr)
-   still reachable: 239 bytes in 8 blocks
+   still reachable: 72,943 bytes in 9 blocks

This is a change that is internal to glibc, perhaps something related to
locale tables.  From memcheck's point of view, the external environment
has changed (using glibc-2.21-13.fc22.x86_64).

This has to do with a change in libstdc++, not glibc.
If the program is linked with libstdc++ from gcc 5.x or higher,
then libstdc++ allocates a block for so called "emergency pool".
This skews the results we expect.
what could be called a false memory leak, the memory block is still
being allocated during the program run.
This particular test case performs incremental leak check analysis
and therefore it does not benefit from fix for 345307.
A solution could be to write a filter.
 

===== massif/tests/new-cpp.post.diff   (post)
-  1          4,008            4,008            4,000             8            0
-  2          8,016            8,016            8,000            16            0
-  3         10,024           10,024           10,000            24            0
-  4         12,032           12,032           12,000            32            0
-  5         12,032           12,032           12,000            32            0
-99.73% (12,000B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
-->33.24% (4,000B) 0x........: main (new-cpp.cpp:19)
+  1         72,712           72,712           72,704             8            0
+  2         76,720           76,720           76,704            16            0
+  3         80,728           80,728           80,704            24            0
+  4         82,736           82,736           82,704            32            0
+  5         84,744           84,744           84,704            40            0
+  6         84,744           84,744           84,704            40            0
+99.95% (84,704B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
+->85.79% (72,704B) 0x........: ??? (in /usr/lib64/libstdc++.so.6.0.21)
+| ->85.79% (72,704B) 0x........: call_init.part.0 (dl-init.c:72)
+|   ->85.79% (72,704B) 0x........: _dl_init (dl-init.c:30)
+|     ->85.79% (72,704B) 0x........: ??? (in /usr/lib64/ld-2.21.so)
+|
+->04.72% (4,000B) 0x........: main (new-cpp.cpp:19)

I believe that this change in output from massif has the same underlying cause
as the change above in the output from memcheck.

Agreed. I tried to fix this in bug 345307 ignoring allocations
done by _GLOBAL__sub_I_eh_alloc.cc. This worked for Linux/Ubuntu 15.10
and Solaris 12. Apparently on this distribution, things are a bit different so another
function needs to be ignored. A little investigation should show what needs to be done here.

Please raise corresponding bugs for these. Patches welcome!
I.



------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Reply | Threaded
Open this post in threaded view
|

Re: expected results for regtest

Mark Roberts

Thank you for your help.  I would like to double check the Valgrind version related to this issue.  I have these same failures on Fedora 22 and Ubuntu 15.10 – but I am using release version 3.11.0 and the date on 345307 is this year.  Sounds like I should retry running from the active svn trunk before reporting any problems.  Is there a particular revision I should sync to or just take the current tree?

 

Thanks again,

Mark

 

From: Ivo Raisr [mailto:[hidden email]]
Sent: Friday, April 15, 2016 12:38 AM
To: John Reiser
Cc: Valgrind Users
Subject: Re: [Valgrind-users] expected results for regtest

 

Hi there,

Thank you for analysing the situation.

Let me clarify things a bit...

 

2016-04-15 2:56 GMT+02:00 John Reiser <[hidden email]>:

===== memcheck/tests/leak_cpp_interior/leak_cpp_interior.stderr.diff-64bit  (stderr)
-   still reachable: 239 bytes in 8 blocks
+   still reachable: 72,943 bytes in 9 blocks

This is a change that is internal to glibc, perhaps something related to
locale tables.  From memcheck's point of view, the external environment
has changed (using glibc-2.21-13.fc22.x86_64).

 

This has to do with a change in libstdc++, not glibc.

If the program is linked with libstdc++ from gcc 5.x or higher,

then libstdc++ allocates a block for so called "emergency pool".

This skews the results we expect.

what could be called a false memory leak, the memory block is still

being allocated during the program run.

This particular test case performs incremental leak check analysis

and therefore it does not benefit from fix for 345307.

A solution could be to write a filter.

 


===== massif/tests/new-cpp.post.diff   (post)
-  1          4,008            4,008            4,000             8            0
-  2          8,016            8,016            8,000            16            0
-  3         10,024           10,024           10,000            24            0
-  4         12,032           12,032           12,000            32            0
-  5         12,032           12,032           12,000            32            0
-99.73% (12,000B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
-->33.24% (4,000B) 0x........: main (new-cpp.cpp:19)
+  1         72,712           72,712           72,704             8            0
+  2         76,720           76,720           76,704            16            0
+  3         80,728           80,728           80,704            24            0
+  4         82,736           82,736           82,704            32            0
+  5         84,744           84,744           84,704            40            0
+  6         84,744           84,744           84,704            40            0
+99.95% (84,704B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
+->85.79% (72,704B) 0x........: ??? (in /usr/lib64/libstdc++.so.6.0.21)
+| ->85.79% (72,704B) 0x........: call_init.part.0 (dl-init.c:72)
+|   ->85.79% (72,704B) 0x........: _dl_init (dl-init.c:30)
+|     ->85.79% (72,704B) 0x........: ??? (in /usr/lib64/ld-2.21.so)
+|
+->04.72% (4,000B) 0x........: main (new-cpp.cpp:19)

I believe that this change in output from massif has the same underlying cause
as the change above in the output from memcheck.

 

Agreed. I tried to fix this in bug 345307 ignoring allocations

done by _GLOBAL__sub_I_eh_alloc.cc. This worked for Linux/Ubuntu 15.10

and Solaris 12. Apparently on this distribution, things are a bit different so another

function needs to be ignored. A little investigation should show what needs to be done here.

 

Please raise corresponding bugs for these. Patches welcome!

I.

 

 


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Reply | Threaded
Open this post in threaded view
|

Re: expected results for regtest

Mark Wielaard
In reply to this post by iraisr
On Fri, Apr 15, 2016 at 09:38:03AM +0200, Ivo Raisr wrote:

> 2016-04-15 2:56 GMT+02:00 John Reiser <[hidden email]>:
> > ===== massif/tests/new-cpp.post.diff   (post)
> > -  1          4,008            4,008            4,000             8
> >     0
> > -  2          8,016            8,016            8,000            16
> >     0
> > -  3         10,024           10,024           10,000            24
> >     0
> > -  4         12,032           12,032           12,000            32
> >     0
> > -  5         12,032           12,032           12,000            32
> >     0
> > -99.73% (12,000B) (heap allocation functions) malloc/new/new[],
> > --alloc-fns, etc.
> > -->33.24% (4,000B) 0x........: main (new-cpp.cpp:19)
> > +  1         72,712           72,712           72,704             8
> >     0
> > +  2         76,720           76,720           76,704            16
> >     0
> > +  3         80,728           80,728           80,704            24
> >     0
> > +  4         82,736           82,736           82,704            32
> >     0
> > +  5         84,744           84,744           84,704            40
> >     0
> > +  6         84,744           84,744           84,704            40
> >     0
> > +99.95% (84,704B) (heap allocation functions) malloc/new/new[],
> > --alloc-fns, etc.
> > +->85.79% (72,704B) 0x........: ??? (in /usr/lib64/libstdc++.so.6.0.21)
> > +| ->85.79% (72,704B) 0x........: call_init.part.0 (dl-init.c:72)
> > +|   ->85.79% (72,704B) 0x........: _dl_init (dl-init.c:30)
> > +|     ->85.79% (72,704B) 0x........: ??? (in /usr/lib64/ld-2.21.so)
> > +|
> > +->04.72% (4,000B) 0x........: main (new-cpp.cpp:19)
> >
> > I believe that this change in output from massif has the same underlying
> > cause
> > as the change above in the output from memcheck.
> >
>
> Agreed. I tried to fix this in bug 345307 ignoring allocations
> done by _GLOBAL__sub_I_eh_alloc.cc. This worked for Linux/Ubuntu 15.10
> and Solaris 12. Apparently on this distribution, things are a bit different
> so another
> function needs to be ignored. A little investigation should show what needs
> to be done here.

The issue I believe is that unless we have the full symtab (from the
separate libstdc++-debuginfo package) we don't know the name of the
function. For memcheck the following suppression seems to work, with
a variant that is stripped and one with full symtab debug symbols:

{
   malloc-leaks-cxx-stl-string-classes
   Memcheck:Leak
   match-leak-kinds: reachable
   fun:malloc
   obj:*/*lib*/libstdc++.so*
   fun:call_init.part.0
   fun:call_init
   fun:_dl_init
   obj:/usr/*lib*/ld-2.*.so
}

{
  malloc-leaks-cxx-stl-string-classes-debug
  Memcheck:Leak
  match-leak-kinds: reachable
  fun:malloc
  fun:pool
  fun:__static_initialization_and_destruction_0
  fun:_GLOBAL__sub_I_eh_alloc.cc
  fun:call_init.part.0
  fun:call_init
  fun:_dl_init
  obj:/usr/*lib*/ld-2.*.so
}

I don't know how to do a similar dual suppression using the massif
--ignore-fn option.

Cheers,

Mark

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Reply | Threaded
Open this post in threaded view
|

Re: expected results for regtest

John Reiser
Mark Wielaard wrote:

> The issue I believe is that unless we have the full symtab (from the
> separate libstdc++-debuginfo package) we don't know the name of the
> function. For memcheck the following suppression seems to work, with
> a variant that is stripped and one with full symtab debug symbols:
>
> {
>     malloc-leaks-cxx-stl-string-classes
>     Memcheck:Leak
>     match-leak-kinds: reachable
>     fun:malloc
>     obj:*/*lib*/libstdc++.so*
>     fun:call_init.part.0
>     fun:call_init
>     fun:_dl_init
>     obj:/usr/*lib*/ld-2.*.so
> }
>
> {
>    malloc-leaks-cxx-stl-string-classes-debug
>    Memcheck:Leak
>    match-leak-kinds: reachable
>    fun:malloc
>    fun:pool
>    fun:__static_initialization_and_destruction_0
>    fun:_GLOBAL__sub_I_eh_alloc.cc
>    fun:call_init.part.0
>    fun:call_init
>    fun:_dl_init
>    obj:/usr/*lib*/ld-2.*.so
> }
>
> I don't know how to do a similar dual suppression using the massif
> --ignore-fn option.

Does the frame-level wildcard "..." work, while still catching real errors?

{
   malloc-leaks-cxx-stl-string-classes-debug
   Memcheck:Leak
   match-leak-kinds: reachable
   fun:malloc
   ...
   fun:call_init.part.0
   fun:call_init
   fun:_dl_init
   obj:/usr/*lib*/ld-2.*.so
}


------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
Reply | Threaded
Open this post in threaded view
|

Re: expected results for regtest

iraisr
In reply to this post by Mark Wielaard

> > +99.95% (84,704B) (heap allocation functions) malloc/new/new[],
> > --alloc-fns, etc.
> > +->85.79% (72,704B) 0x........: ??? (in /usr/lib64/libstdc++.so.6.0.21)
> > +| ->85.79% (72,704B) 0x........: call_init.part.0 (dl-init.c:72)
> > +|   ->85.79% (72,704B) 0x........: _dl_init (dl-init.c:30)
> > +|     ->85.79% (72,704B) 0x........: ??? (in /usr/lib64/ld-2.21.so)
> > +|
> > +->04.72% (4,000B) 0x........: main (new-cpp.cpp:19)
> >
> > I believe that this change in output from massif has the same underlying
> > cause
> > as the change above in the output from memcheck.
> >
>
> Agreed. I tried to fix this in bug 345307 ignoring allocations
> done by _GLOBAL__sub_I_eh_alloc.cc. This worked for Linux/Ubuntu 15.10
> and Solaris 12. Apparently on this distribution, things are a bit different
> so another
> function needs to be ignored. A little investigation should show what needs
> to be done here.

The issue I believe is that unless we have the full symtab (from the
separate libstdc++-debuginfo package) we don't know the name of the
function. For memcheck the following suppression seems to work, with
a variant that is stripped and one with full symtab debug symbols:

{
   malloc-leaks-cxx-stl-string-classes
   Memcheck:Leak
   match-leak-kinds: reachable
   fun:malloc
   obj:*/*lib*/libstdc++.so*
   fun:call_init.part.0
   fun:call_init
   fun:_dl_init
   obj:/usr/*lib*/ld-2.*.so
}

{
  malloc-leaks-cxx-stl-string-classes-debug
  Memcheck:Leak
  match-leak-kinds: reachable
  fun:malloc
  fun:pool
  fun:__static_initialization_and_destruction_0
  fun:_GLOBAL__sub_I_eh_alloc.cc
  fun:call_init.part.0
  fun:call_init
  fun:_dl_init
  obj:/usr/*lib*/ld-2.*.so
}

I don't know how to do a similar dual suppression using the massif
--ignore-fn option.

Could you try: --ignore-fn=call_init.part.0
Cheers,
I.

------------------------------------------------------------------------------
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers of
your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
Valgrind-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-users