Migrating Valgrind sources from SVN to GIT

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Migrating Valgrind sources from SVN to GIT

iraisr
Dear Valgrind community,

Now that Valgrind release 3.13.0 is out, we are pleased to announce
an imminent migration of Valgrind sources from existing Subversion SCM
to modern git SCM, as discussed during our FOSDEM 2017 Valgrind devroom
and announced publicly back at the end of February 2017.

What is going on now?
~~~~~~~~~~~~~~~~~
The migration is bound to happen in a few weeks. We are now in the last
phase of beta testing stage.
We still use the official SVN Valgrind repository for our work until
the final migration step. If you have some patches ready now, send
them for review.
You can contribute to the migration process - read below.

What will be migrated:
~~~~~~~~~~~~~~~~~
Valgrind and VEX sources. Precisely sources available today under
svn://svn.valgrind.org/valgrind and svn://svn.valgrind.org/vex, including
all production release branches and tags. Valgrind and VEX repos will
be merged into one, so no more SVN externals.

Where I will find the new repo:
~~~~~~~~~~~~~~~~~~~~~~~
At sourceware.org. Precisely at:
git://sourceware.org/git/valgrind.git/
http://sourceware.org/git/valgrind.git/
Right now a snapshot of SVN sources as of 2017-06-16 is available for
you to test.

How the test migration was performed:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
See recipes at https://github.com/ivosh/valgrind-git-migration

What is the plan for the migration to go forward:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Test migration has been performed and initial tests were successful.
2. The test repo is now available to test and play for others with - see below
    for details.
3. www (website) and nightly build script changes have been prepared
    in https://github.com/ivosh/valgrind-git-migration.
    They need to be reviewed.
4. Proceed once 2+3 are successfully done.
5. Announce the final migration.
6. Completely eradicate contents in the GIT repository so the migration
    can start from scratch.
7. Switch SVN valgrind+vex repo readonly.
8. Perform the final migration to sourceware.org.
9. Enable email notifications from new git repo.
10. Push www and nightly script changes to the new repo.

What will not be migrated:
~~~~~~~~~~~~~~~~~~~~
- Valgrind www (website) repo. Not now, but later.
- Non production release branches and tags from old SVN Valgrind+VEX repos.
  If you need to preserve some other branches or tags, let us know:
  https://sourceware.org/git/?p=valgrind.git;a=heads
  https://sourceware.org/git/?p=valgrind.git;a=tags

I have a write access to existing SVN repo. What shall I do for the new one?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Please contact Julian Seward. He will point you to specific instructions.

What will be my simple workflow in new git SCM?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Not much will be changed from the way we worked in SVN.
We still prepare patches, send them for review, have someone
with write access to push them. A minimalistic workflow would be:

git clone git://sourceware.org/git/valgrind.git/ valgrind
edit/compile
git status/add/show
git pull --rebase origin/master
build + test
git commit
[git push - if you have write access]

There are a lot of good tutorials on simple git workflows, so please
have a look. If you are using something more complicated, please
share with us and ideally send us a write up.

I would like to help with the migration.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Yes, please! Send us your positive and negative feedback.
For example:
- It worked for me!
- This and that did not work for me...
- How do I do such and such thing now?

The test repository is there for you to play with. The contents
will be deleted before the final migration so no reason to worry about
potential mistakes.

We also need a help documenting possible workflows. Especially when
preparing a release - we need to test and document how to work with branches
and releases.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Valgrind-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-developers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating Valgrind sources from SVN to GIT

iraisr
2017-06-21 4:05 GMT+02:00 Bart Van Assche <[hidden email]>:

> On 06/16/17 08:17, Ivo Raisr wrote:
>>
>> What will be my simple workflow in new git SCM?
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> Not much will be changed from the way we worked in SVN.
>> We still prepare patches, send them for review, have someone
>> with write access to push them. A minimalistic workflow would be:
>>
>> git clone git://sourceware.org/git/valgrind.git/ valgrind
>> edit/compile
>> git status/add/show
>> git pull --rebase origin/master
>> build + test
>> git commit
>> [git push - if you have write access]
>>
>> There are a lot of good tutorials on simple git workflows, so please
>> have a look. If you are using something more complicated, please
>> share with us and ideally send us a write up.
>
>
> Hello Ivo,
>
> Thank you for your detailed and very informative e-mail. As far as I know
> the current approach for sharing and reviewing patches is to create a
> bugzilla ticket, attach the patch to the bugzilla ticket and address any
> review comments that are posted in the bugzilla ticket. Servers like github
> and gitlab support a more convenient approach, namely that a developer
> clones a project into his or her personal space, publishes patches in that
> space and then sends a pull request to the maintainers. This is more
> convenient than the bugzilla attachment approach because it requires less
> work for developers and less work for reviewers. Do you perhaps know whether
> the sourceware.org server supports submitting pull requests and reviewing
> pull requests?


Hello Bart,

Thank you for raising this question.
Today this topic was discussed on IRC channel, largely between Mark and Tom.
Let me paste the whole conversation below, I find it quite interesting.
Nevertheless, to answer your question, sourceware.org does not offer
any web based interface
for submitting and reviewing pull requests. Other GNU projects, such as
gdb, binutils, elfutils, glibc, follow basically very similar approach
as Valgrind will
do after git migration.
On the other hand, if majority of Valgrind developers would like to
migrate to github
or gitlab, and Julian agrees, then I can prepare migration path to
such an environment.

Kind regards,
I.





-------------------------------------
(13:05:24) Ivosh: mjw: Bart asks if sourceware.org supports
pull-request workflow, known for example from bitbucket or github
(13:05:30) Ivosh: mjw: any idea?
(13:06:56) tomhughes: well that should be fairly easy to determine -
just look at the web site!
(13:07:07) tomhughes: but basically no
(13:07:20) tomhughes: as the 1995 vintage homepage might hint ;-)
(13:08:39) tomhughes: gitweb is all it offers in terms of a web
interface by the looks ofit
(13:20:08) sewardj: Ivosh: ach .. I had a request from Bart re github,
and forgot to forward it to you
(13:22:15) uweigand [uweigand@nat/ibm/x-uaahwvoirxjslfxe] entered the room.
(13:23:44) arnez left the room.
(13:26:47) mjw: Ivosh, It supports real git pull requests
(13:26:55) mjw: But not the web based thingy
(13:27:04) mjw: Which I never could make to work myself btw.
(13:28:03) mjw: What we do for elfutils, which is on sourceware. Is
have people have personal branches like mjw/dwarf5-line-table
(13:29:20) tomhughes: what is a "real git pull request"
(13:29:35) tomhughes: do you just mean an email asking somebody to
pull from a repo?
(13:29:42) tomhughes: I mean it's hard not to support that!
(13:30:06) mjw: then you can either just point people at that, git
send-email the changes from the master branch or do a formal git
request-pull
(13:32:14) mjw: tomhughes, yes, I do believe there are tools for
turning the web based github pull request into what basically looks
like a normal git request-pull
(13:33:22) mjw: Maybe those can be used without having github account.
But I found github too confusing. I must be missing something because
I have seen others use it just fine.
(13:34:06) mjw: It might be my reluctance to fire up a browser just to
do something simple.
(13:34:53) tomhughes: I've never really used anything other than
github - there's a few tricks to make PRs even easier to work with but
even without those it's dead simple
(13:35:02) tomhughes: Much easier than dealing with format-patch
(13:35:06) tomhughes: and emails
(13:35:26) tomhughes: I have no idea what request-pull even does
(13:35:27) mjw: really? That is all I ever use git send-email; git am
(13:35:50) tomhughes: well that's easy for you, but massively hard
work for the maintainer
(13:36:13) mjw: tomhughes, ? I am the maintainer (well for elfutils)
(13:36:33) tomhughes: I have to save the email out of thunderbird,
copy it to the right machine and then hope nothing has mangled it too
much
(13:36:40) mjw: ehe?
(13:36:58) mjw: I mainly use mutt, but I assume with thunderbird you
can also | git am
(13:37:22) tomhughes: as against "git fetch --all; git merge
openstreetmap/pull/994" for a github PR on the osm code
(13:37:37) mjw: that is really all there is to it. Look at it, do a
make check or something, reply to the email with the review or git
merge to master and push.
(13:37:38) tomhughes: I know of no way to pipe anything from thunderbirs
(13:37:56) tomhughes: and anyway if it's the office or laptop
thunderbird and the repo is on my desktop at home
(13:38:27) mjw: ah, that is a pity. Not having to leave your email
environment for simple patch apply/review is a big plus (to me)
(13:38:52) tomhughes: plus the web review interface is way nicer than
trying to read and comment on a patch in an email client
(13:39:12) tomhughes: I have no idea how the linux kernel folks get
anything done
(13:39:21) mjw: That might be a personal preference indeed
(13:39:25) tomhughes: I find it impossible to even read threads where
they are revireing patches
(13:39:44) mjw: I git completely lost in the browser, ponty-clicky
style "review" that is github.
(13:39:51) mjw: and the emails it sends are basically useless :{
(13:40:06) mjw: Interesting
(13:40:45) tomhughes: BTW the one secret hack I use is to add eg
"fetch = +refs/pull/*/head:refs/remotes/openstreetmap/pull/*" to the
github remote in .git/config so that it pulls down PRs as extra heads
for easy merging
(13:41:11) mjw: I do believe you. I am just surprised. I find the
email based workflow where you have everything together in the
relevant threads and don't need any browser/network access far
superior for my workflow.
(13:42:31) mjw: tomhughes, that might be the trick I was missing.
Looks like with that it becomes basically the user/branch workflow.
You pull all branches and then can review/apply as normal.
(13:44:14) mjw: I just don't understand how you do proper review
without using email. But that might just be because I find it natural
that (I am used to) a patch code flow equals a email conversation.
(13:44:16) tomhughes: or you can do something like
https://gist.github.com/karlhorky/0c454c0c6f894c27911ed5de58d65416 to
add a fetch-pr alias that just grabs one
(13:44:38) tomhughes: basically it's knowing they have the
refs/pull/NNN heads on the remote that you can do things with
(13:44:43) mjw: I don't really like the valgrind bugzilla workflow
either btw. But heay, that is how things are done. I'll adjust :)
(13:45:55) tomhughes: which they sort of document in
https://help.github.com/articles/checking-out-pull-requests-locally/
but only show you the slowest and most painful ways to use it ;-)
(13:46:56) mjw: I did like gerrit based workflows btw. Because you can
do anything with them just using such git tricks. And only needed the
website thingy for registering a user name.
(13:47:24) mjw: and I also heard good things about pagure and gitlab,
though I only very briefly used the former.
(13:47:41) tomhughes: I have one repo on paguro.io
(13:47:51) tomhughes: but very much looking forward to pague over distgit
(13:48:08) mjw: Those at least you can run yourself because they are
free software. The github thing is all proprietary and doesn't really
federates.
(13:48:56) tomhughes: oh sure I have quesiness about the specifics og
github as a closed source system
(13:49:15) tomhughes: but they do a good job and obviously benefit
from a network effect
(13:49:23) tomhughes: gitlad or pagure offer very similar though
(13:49:30) mjw: Then again, we (valgrind) use sourceforge for our mailinglist :)
(13:49:38) tomhughes: argh don't remind me
(13:50:03) tomhughes: and bugzilla for our issue tracker ;-)
(13:50:11) mjw: BTW. I am sure the sourceware admins would be fine
with us using it for email too. If people want.
(13:50:27) arnez [arnez@nat/ibm/x-lacatgtfuglwlnus] entered the room.
(13:50:27) tomhughes: do they have a less insane archives interface...
(13:50:52) tomhughes: simple answer: yes
(13:50:55) mjw: Only thing to be aware of is that all their lists all
open to everybody (their spam filters are really good, partly because
they just reject any HTML email).
(13:50:58) tomhughes: mainly because it would be hard not to
(13:51:21) tomhughes: err right, that's more of a user filter than a
spam filter these days?
(13:51:30) mjw: indeed :)
(13:51:34) tomhughes: or do you mean they strip the html from
multipart/alternative
(13:52:05) mjw: no, they require text/plain emails
(13:52:24) mjw: that is indeed a pretty good filter to only get developers
(13:52:40) mjw: It is less ideal for a user bases communications channel.
(13:53:43) tomhughes: it would reject the last two end-user emails to
valgrind-users for example ;-)
(13:54:08) mjw: tomhughes, BTW. When you say dist-git over pagure. Do
you mean that the fedora package repos will be moved to a pagure based
setup?
(13:54:36) tomhughes: err yes - in the next few weeks
(13:54:45) mjw: ah...
(13:54:59) ***mjw hides that he sometimes isn't paying enough
attention to fedora :)
(13:55:40) tomhughes: as part of
https://fedoraproject.org/wiki/Changes/ArbitraryBranching
(13:55:44) mjw: Which reminds me I should push and email about some
upstream rpm improvements that are coming for f27...
(13:55:52) tomhughes: well it was planned anyway but it's necessary to
do it now for that
(13:56:06) tomhughes: Julth 5th is current target
(13:57:08) mjw: fedora is sometimes hard to keep track of. Too many
places for communication and fedora-devel is a bit too high volume to
keep reading daily.
(13:58:40) mjw: This change request is a good example, I am reading
it, but still don't know what it is about :)
(13:59:49) mjw: O. I get it. We get more branches to build from.
(13:59:59) mjw: It turns dist-git into copr?
(14:00:50) mjw: I like copr. So if we just pretend that is this, then
I am all for it :)
(14:04:43) tomhughes: oh I'm unconvinced abouyt arbitrary branching
(14:05:11) tomhughes: basically you can choose your branch names and
then build the same branch into multiple koji targets if you want
(14:05:33) mjw: which is what the modularity people seem to do
(14:05:34) tomhughes: so you can have valgrind-3.11 and valgrind-3.12
as branches instead of f424, f25, f26 etc
(14:05:43) tomhughes: yeah I'm ingorning modularity
(14:06:01) mjw: which is what I am doing on copr now (kind of, except
no underlying branch, just a srpm)
(14:06:04) tomhughes: as much as possible at any rate
(14:07:53) mjw: I am happy I have a spec now that builds unchanged on
f2x, centos 6/7, dts, etc. That really makes my life a lot easier and
I can pull in feedback from various different users/use cases.
(14:08:53) mjw: I certainly see the appeal of "modularity" where you
don't care whether it is f24 or f27, but just focus on your package
and let your users pull in the one they need, instead of you
pushing/splitting things into fxx, fxy, fyy, etc.
(14:10:23) tomhughes: well as an end user I don't really want to be
having to decide what eversion of everything install on a case-by-case
basis...
(14:10:42) tomhughes: I just want to be able to update and get the
latest of everything
(14:11:10) tomhughes: nor as a maintainer do I want the extra work of
maintaining multiple version
(14:12:17) tuliom [~quassel@187.10.26.201] entered the room.
(14:28:57) mjw: tomhughes, yes, I think my viewpoint might be skewed a
bit by being responsible to keep the latest working on older RHEL.
(14:29:13) mjw: If I wasn't paid for that I might indeed not care myself...
(14:30:59) tomhughes: yeah I don't bother with RHEL for any of my packages ;-)
(14:31:52) Ivosh: hmm, interesting conversation
(14:32:32) mjw: On the other hand. I do run RHEL7 on my workstation. I
love it being super stable (read, old and boring :)
(14:33:11) mjw: It does let me focus on what I really care about
without having to worry about updates messing up integration issues.
(14:33:47) mjw: I do of course run Fedora on my laptop to keep up with
the latest and greatest. But for daily development I use RHEL.
(14:35:12) mjw: (You may of course replace RHEL with the latest Debian
stable, to get the same experience - although I do believe RHEL - or
CentOS - is actually a bit more stable and developer friendly - that
might be my employment bias though...)
(14:40:45) Ivosh: actually I run Ubuntu on my laptop and for testing
patches I download them directly from bugzilla via wget

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Valgrind-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-developers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating Valgrind sources from SVN to GIT

bart.vanassche (Bugzilla)
On Wed, Jun 21, 2017 at 6:03 AM, Ivo Raisr <[hidden email]> wrote:
> (13:37:38) tomhughes: I know of no way to pipe anything from thunderbirs
> (13:37:56) tomhughes: and anyway if it's the office or laptop
> thunderbird and the repo is on my desktop at home
> (13:38:27) mjw: ah, that is a pity. Not having to leave your email
> environment for simple patch apply/review is a big plus (to me)
> (13:38:52) tomhughes: plus the web review interface is way nicer than
> trying to read and comment on a patch in an email client
> (13:39:12) tomhughes: I have no idea how the linux kernel folks get
> anything done

Hello Tom and Mark,

Every kernel developer I know uses an e-mail client that supports
e-mail threading. Without an e-mail client that supports threading
it's very hard to follow patch review discussions.

Regarding the workflow of kernel maintainers: some use mutt as e-mail
client. Others use a GUI e-mail client and use patchwork for managing
patches. Patchwork is a server that picks up patches and patch series
from Linux kernel related mailing lists. But I agree that from a
maintainer point-of-view github and gitlab are far easier to use than
the tools used by kernel developers and maintainers. See also
https://patchwork.kernel.org/.

Bart.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Valgrind-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-developers
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Migrating Valgrind sources from SVN to GIT

iraisr
2017-06-21 18:51 GMT+02:00 Bart Van Assche <[hidden email]>:
> Regarding the workflow of kernel maintainers: some use mutt as e-mail
> client. Others use a GUI e-mail client and use patchwork for managing
> patches. Patchwork is a server that picks up patches and patch series
> from Linux kernel related mailing lists. But I agree that from a
> maintainer point-of-view github and gitlab are far easier to use than
> the tools used by kernel developers and maintainers. See also
> https://patchwork.kernel.org/.

I had a look at Patchwork system and it looks quite nice!
Surprisingly, there is a patchwork instance running at sourceware.org:
https://patchwork.sourceware.org/

So if the community agrees, we can start using patchwork to track our patches.
However this is really orthogonal to what SCM Valgrind uses.

I.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Valgrind-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/valgrind-developers
Loading...