librelist archives

« back to archive

Priorities (of source, wheel, local and downloaded)

Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-24 @ 13:48
I've been doing some tests. The experiment below worries me.

It seems to me that this is saying that even though I specify
--use-wheel, and I have a wheel in a local find-links directory, pip
still decides to download a sdist and build it. I can't see how this
could be right. Surely the precedence should be (assuming the same
version everywhere)

1. Local files take precedence over internet downloads
2. Wheels take precedence over sdists

The second of these in particular is a problem. If I'm on a system
where I can't build from source (no compiler, say) I have no means of
using wheels without doing --no-index, which means that if I've missed
putting a dependency in my wheel cache, the install fails rather than
downloading it.

I'd appreciate comments on this - am I misunderstanding? Or is there a
use case I'm not appreciating?

Paul.

>pip install -f . --use-wheel bitarray
Downloading/unpacking bitarray
  Using download cache from

D:\PipDownload\http%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fb%2Fbitarray%2Fbitarray-0.8.0.tar.gz
  Running setup.py egg_info for package bitarray

Installing collected packages: bitarray
  Running setup.py install for bitarray
    building 'bitarray._bitarray' extension
    C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\cl.exe /c
/nologo /Ox /MD /W3 /GS- /DNDEBUG -ID:\Apps\Python33\include
-ID:\Apps\Python33\include /Tcbitarray/_bitarray.c
/Fobuild\temp.win32-3.3\Release\bitarray/_bitarray.obj
    _bitarray.c
    C:\Program Files\Microsoft Visual Studio 10.0\VC\BIN\link.exe /DLL
/nologo /INCREMENTAL:NO /LIBPATH:D:\Apps\Python33\Libs
/LIBPATH:D:\Dev\v33\libs /LIBPATH:D:\Dev\v33\PCbuild
/EXPORT:PyInit__bitarray
build\temp.win32-3.3\Release\bitarray/_bitarray.obj
/OUT:build\lib.win32-3.3\bitarray\_bitarray.pyd
/IMPLIB:build\temp.win32-3.3\Release\bitarray\_bitarray.lib
/MANIFESTFILE:build\temp.win32-3.3\Release\bitarray\_bitarray.pyd.manifest
       Creating library
build\temp.win32-3.3\Release\bitarray\_bitarray.lib and object
build\temp.win32-3.3\Release\bitarray\_bitarray.exp

Successfully installed bitarray
Cleaning up...
(v33) PS 14:42 D:\Dev\tt
>dir


    Directory: Microsoft.PowerShell.Core\FileSystem::D:\Dev\tt


Mode           LastWriteTime       Length Name
----           -------------       ------ ----
-a---    22/10/2012    22:28        39464 bitarray-0.8.0-cp33-none-win32.whl

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-24 @ 17:31
On Wed, Oct 24, 2012 at 6:48 AM, Paul Moore <p.f.moore@gmail.com> wrote:

> I've been doing some tests. The experiment below worries me.
>
> It seems to me that this is saying that even though I specify
> --use-wheel, and I have a wheel in a local find-links directory, pip
> still decides to download a sdist and build it. I can't see how this
> could be right. Surely the precedence should be (assuming the same
> version everywhere)
>
> 1. Local files take precedence over internet downloads
> 2. Wheels take precedence over sdists
>
> The second of these in particular is a problem. If I'm on a system
> where I can't build from source (no compiler, say) I have no means of
> using wheels without doing --no-index, which means that if I've missed
> putting a dependency in my wheel cache, the install fails rather than
> downloading it.
>
> I'd appreciate comments on this - am I misunderstanding? Or is there a
> use case I'm not appreciating?
>

this is the "wheel prioritization" issue that I was going to work on.
also, note this pull in the real pip: https://github.com/pypa/pip/pull/702

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-24 @ 17:53
On 24 October 2012 18:31, Marcus Smith <qwcode@gmail.com> wrote:
>> I'd appreciate comments on this - am I misunderstanding? Or is there a
>> use case I'm not appreciating?
>
> this is the "wheel prioritization" issue that I was going to work on.
> also, note this pull in the real pip: https://github.com/pypa/pip/pull/702

OK, that's cool. I knew there had been mention of this, but I'd
thought you were talking about the more complicated issues that Daniel
mentioned in his reply, not the basic "wheels before sdists".

I can see if I can get some time to look at it, as well - as it's key
to me, I think I should do a bit more than just sit back and wait for
you to fix it :-) (Although I'd be happy if you did!)
Paul

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-24 @ 17:59
>
> I can see if I can get some time to look at it, as well - as it's key
> to me, I think I should do a bit more than just sit back and wait for
> you to fix it :-) (Although I'd be happy if you did!)


feel free to take a crack at it.  : )

this is the desired sort scheme, right?

1st) version (when not explicity specified)
2nd) wheel over sdist (assuming "--prefer-wheel")
3rd) installed over local over index links over dependency links over
"egg=" links
    (see: https://github.com/pypa/pip/pull/702)

it was imagining just using sorted against a list like this

links = [
  (version, dist_type, link_type)
  (version, dist_type, link_type)
   ....
]

where dist_type and link_type are values from an "enumeration"

dist_types:
  DIST_WHEEL = 2
  DIST_SOURCE = 1

link_types:
  LINK_INSTALLED = 1
  LINK_LOCAL = 2
  etc...

I'd really like anyone who attempts this, to try to clean up the finder for
readability, and not just tack this on though.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-25 @ 18:35
On 24 October 2012 18:59, Marcus Smith <qwcode@gmail.com> wrote:
>> I can see if I can get some time to look at it, as well - as it's key
>> to me, I think I should do a bit more than just sit back and wait for
>> you to fix it :-) (Although I'd be happy if you did!)
>
>
> feel free to take a crack at it.  : )
>
> this is the desired sort scheme, right?
>
> 1st) version (when not explicity specified)
> 2nd) wheel over sdist (assuming "--prefer-wheel")
> 3rd) installed over local over index links over dependency links over "egg="
> links
>     (see: https://github.com/pypa/pip/pull/702)
>
> it was imagining just using sorted against a list like this
>
> links = [
>   (version, dist_type, link_type)
>   (version, dist_type, link_type)
>    ....
> ]
>
> where dist_type and link_type are values from an "enumeration"
>
> dist_types:
>   DIST_WHEEL = 2
>   DIST_SOURCE = 1
>
> link_types:
>   LINK_INSTALLED = 1
>   LINK_LOCAL = 2
>   etc...

OK. One question, what should I base my patch against? Your
wheel_install branch is the obvious answer, but the main pip repo now
has pull 702 incorporated, which will conflict with any changes here.
I'm far from being a git master, so I'm unsure what's best. If I were
guessing, I'd do the following (starting from my fork of pypa/git,
with a remote upstream for pypa and one qwcode which is your fork):

git checkout develop
git merge upstream/develop
git checkout -b finder_priority
git merge qwcode/wheel_install
# hack on this branch
# Create a pull request for you from this branch

Would that make sense?

> I'd really like anyone who attempts this, to try to clean up the finder for
> readability, and not just tack this on though.

Great! Having the chance to do a bit of refactoring as part of trying
to make sense of the code helps me.

There's one comment that worries me, though - _link_package_versions
says "Meant to be overridden by subclasses, not called by clients".
The PackageFinder class is not overridden anywhere in the pip codebase
that I can see. So I'm concerned - is there an intention that external
code might subclass PackageFinder? If so, it's pretty crucial that any
changes don't change the API that subclasses expect (and so, things
like _link_package_versions have to stay). But given that there's no
documented interface for subclassing, this quickly leads to a point
where suclasses can assume anything, and the whole thing gets stuck
from compatibility concerns.

My instinct is to assume that this comment is not actually
significant, and that no subclasses of PackageFinder exist "in the
wild". So it's (reasonably) OK to make changes - although obviously I
don't intend to do so gratuitously.

What do you think?

Paul.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-25 @ 18:59
>
>
> OK. One question, what should I base my patch against? Your
> wheel_install branch is the obvious answer, but the main pip repo now
> has pull 702 incorporated, which will conflict with any changes here.
> I'm far from being a git master, so I'm unsure what's best. If I were
> guessing, I'd do the following (starting from my fork of pypa/git,
> with a remote upstream for pypa and one qwcode which is your fork):
>
> git checkout develop
> git merge upstream/develop
> git checkout -b finder_priority
> git merge qwcode/wheel_install
> # hack on this branch
> # Create a pull request for you from this branch
>

let me go rebase against the latest right now, and you want have to worry
with it.
I'll email back in a minute.


> My instinct is to assume that this comment is not actually
> significant, and that no subclasses of PackageFinder exist "in the
> wild". So it's (reasonably) OK to make changes


agreed.  I saw the subclass comment as well.  once you get in there, just
make the best judgement I guess.

Marcus

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-25 @ 19:10
>
> let me go rebase against the latest right now, and you want have to worry
> with it.
> I'll email back in a minute.
>

I tend to be rebase-biased, but if you think that's a bad idea, let me know.
while we're working, I'd just to prefer to keep our commits showing on the
top after I periodically integrate with the main pip.

Marcus

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-25 @ 19:54
On Thu, Oct 25, 2012 at 12:10 PM, Marcus Smith <qwcode@gmail.com> wrote:

> let me go rebase against the latest right now, and you want have to worry
>> with it.
>> I'll email back in a minute.
>>
>
> I tend to be rebase-biased, but if you think that's a bad idea, let me
> know.
> while we're working, I'd just to prefer to keep our commits showing on the
> top after I periodically integrate with the main pip.
>
>
on 2nd thought rebasing on pypa/pip:develop upstream is *bad* idea.
I temporarily had force pushed up a rebase for a few minutes, but now
what's up there is just normal merge with pypa/develop
https://github.com/qwcode/pip




> Marcus
>

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-25 @ 20:08
>
>
> on 2nd thought rebasing on pypa/pip:develop upstream is *bad* idea.
> I temporarily had force pushed up a rebase for a few minutes, but now
> what's up there is just normal merge with pypa/develop
> https://github.com/qwcode/pip
>
>
so, paul,  to be clear, all you should need to do is update your local
wheel_install branch, then  branch from there.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-24 @ 18:07
>
> 3rd) installed over local over index links over dependency links over
> "egg=" links
>     (see: https://github.com/pypa/pip/pull/702)
>
>
btw, paul since you've got your mind on this right now... a review of pull
#702 would be awesome.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-25 @ 10:19
On 24 October 2012 19:07, Marcus Smith <qwcode@gmail.com> wrote:
>> 3rd) installed over local over index links over dependency links over
>> "egg=" links
>>     (see: https://github.com/pypa/pip/pull/702)
>>
>
> btw, paul since you've got your mind on this right now... a review of pull
> #702 would be awesome.

Looks like it got merged before I got to it. Sorry about that...
Paul

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-25 @ 15:47
the other Paul got to it  :  )
I had asked him to look at it too.

On Thu, Oct 25, 2012 at 3:19 AM, Paul Moore <p.f.moore@gmail.com> wrote:

> On 24 October 2012 19:07, Marcus Smith <qwcode@gmail.com> wrote:
> >> 3rd) installed over local over index links over dependency links over
> >> "egg=" links
> >>     (see: https://github.com/pypa/pip/pull/702)
> >>
> >
> > btw, paul since you've got your mind on this right now... a review of
> pull
> > #702 would be awesome.
>
> Looks like it got merged before I got to it. Sorry about that...
> Paul
>

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-24 @ 18:14
On 24 October 2012 19:07, Marcus Smith <qwcode@gmail.com> wrote:
>> 3rd) installed over local over index links over dependency links over
>> "egg=" links
>>     (see: https://github.com/pypa/pip/pull/702)
>>
>
> btw, paul since you've got your mind on this right now... a review of pull
> #702 would be awesome.

I'm out this evening but I'll have a look tomorrow.
Paul

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-24 @ 19:07
>
> this is the desired sort scheme, right?
>
> 1st) version (when not explicity specified)
> 2nd) wheel over sdist (assuming "--prefer-wheel")
> 3rd) installed over local over index links over dependency links over
> "egg=" links
>     (see: https://github.com/pypa/pip/pull/702)
>
>
actually, to be more precise...

1st) version
2nd) wheel over sdist (assuming "--prefer-wheel")
3rd) index/find-link over dependency link
4th) normal over "egg=" links
     (see http://packages.python.org/distribute/setuptools.html
"distributed as a single .py file, you must include an
"#egg=project-version" )
5th) local over remote

crazy?

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-25 @ 18:02
On 24 October 2012 20:07, Marcus Smith <qwcode@gmail.com> wrote:
>> this is the desired sort scheme, right?
>>
>> 1st) version (when not explicity specified)
>> 2nd) wheel over sdist (assuming "--prefer-wheel")
>> 3rd) installed over local over index links over dependency links over
>> "egg=" links
>>     (see: https://github.com/pypa/pip/pull/702)
>>
>
> actually, to be more precise...
>
> 1st) version
> 2nd) wheel over sdist (assuming "--prefer-wheel")
> 3rd) index/find-link over dependency link
> 4th) normal over "egg=" links
>      (see http://packages.python.org/distribute/setuptools.html
> "distributed as a single .py file, you must include an
> "#egg=project-version" )
> 5th) local over remote
>
> crazy?

Do you really feel the need for a --prefer-wheel flag? Given that you
have to explicitly say --use-wheel at the moment, I'd suggest that
implies prefer as well. Later, if we make --use-wheel the default,
then maybe rename the option to --prefer-wheel and have it default to
off.

Paul.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-25 @ 18:31
I don't want 2 options.
whatever we call it, here's what I think it should mean

"--XXXX-wheel (default=false)":
     - if a wheel is available anywhere, use it
     - if a wheel is not available, then use an sdist if you can

at some in the *future*,  the default could be true

On Thu, Oct 25, 2012 at 11:02 AM, Paul Moore <p.f.moore@gmail.com> wrote:

> On 24 October 2012 20:07, Marcus Smith <qwcode@gmail.com> wrote:
> >> this is the desired sort scheme, right?
> >>
> >> 1st) version (when not explicity specified)
> >> 2nd) wheel over sdist (assuming "--prefer-wheel")
> >> 3rd) installed over local over index links over dependency links over
> >> "egg=" links
> >>     (see: https://github.com/pypa/pip/pull/702)
> >>
> >
> > actually, to be more precise...
> >
> > 1st) version
> > 2nd) wheel over sdist (assuming "--prefer-wheel")
> > 3rd) index/find-link over dependency link
> > 4th) normal over "egg=" links
> >      (see http://packages.python.org/distribute/setuptools.html
> > "distributed as a single .py file, you must include an
> > "#egg=project-version" )
> > 5th) local over remote
> >
> > crazy?
>
> Do you really feel the need for a --prefer-wheel flag? Given that you
> have to explicitly say --use-wheel at the moment, I'd suggest that
> implies prefer as well. Later, if we make --use-wheel the default,
> then maybe rename the option to --prefer-wheel and have it default to
> off.
>
> Paul.
>

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-25 @ 18:36
On 25 October 2012 19:31, Marcus Smith <qwcode@gmail.com> wrote:
> I don't want 2 options.
> whatever we call it, here's what I think it should mean
>
> "--XXXX-wheel (default=false)":
>      - if a wheel is available anywhere, use it
>      - if a wheel is not available, then use an sdist if you can
>
> at some in the *future*,  the default could be true

OK, that's what I thought. I see no reason to change the option name
from --use-wheel right now. Let's paint that bikeshed later :-)

Paul.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-25 @ 18:57
On Thu, Oct 25, 2012 at 11:36 AM, Paul Moore <p.f.moore@gmail.com> wrote:

> On 25 October 2012 19:31, Marcus Smith <qwcode@gmail.com> wrote:
> > I don't want 2 options.
> > whatever we call it, here's what I think it should mean
> >
> > "--XXXX-wheel (default=false)":
> >      - if a wheel is available anywhere, use it
> >      - if a wheel is not available, then use an sdist if you can
> >
> > at some in the *future*,  the default could be true
>
> OK, that's what I thought. I see no reason to change the option name
> from --use-wheel right now. Let's paint that bikeshed later :-)
>

cool with me.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Daniel Holth
Date:
2012-10-24 @ 14:04
On Wed, Oct 24, 2012 at 9:48 AM, Paul Moore <p.f.moore@gmail.com> wrote:
> I've been doing some tests. The experiment below worries me.
>
> It seems to me that this is saying that even though I specify
> --use-wheel, and I have a wheel in a local find-links directory, pip
> still decides to download a sdist and build it. I can't see how this
> could be right. Surely the precedence should be (assuming the same
> version everywhere)
>
> 1. Local files take precedence over internet downloads
> 2. Wheels take precedence over sdists
>
> The second of these in particular is a problem. If I'm on a system
> where I can't build from source (no compiler, say) I have no means of
> using wheels without doing --no-index, which means that if I've missed
> putting a dependency in my wheel cache, the install fails rather than
> downloading it.
>
> I'd appreciate comments on this - am I misunderstanding? Or is there a
> use case I'm not appreciating?

The precedence concept exists in egg as well:

https://bitbucket.org/tarek/distribute/src/tip/pkg_resources.py?at=default#cl-191

EGG_DIST    = 3
BINARY_DIST = 2
SOURCE_DIST = 1
CHECKOUT_DIST = 0
DEVELOP_DIST = -1

I would like to see the wheel ranking scheme extended to include any
kind of dist. Imagine a cleanly sortable Dist() or LinkToDist() (bad
names) that includes all kinds of dists including wheels for choosing
the best from a particular version.

I think people will be very happy if there is a flag --prefer-wheel /
--prefer-sdist. The extended ranking used to sort dists having the
same version would be, depending on the flag,

sdist
other-kinds-of-dist?
cp33-none-win32
py3-none-any

or

cp33-none-win32
py3-none-any
sdist
other-kinds-of-dist?

Can there be a --prefer-local-wheel-remote-sdist without confusing everyone?

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-24 @ 17:01
On 24 October 2012 15:04, Daniel Holth <dholth@gmail.com> wrote:
>> I'd appreciate comments on this - am I misunderstanding? Or is there a
>> use case I'm not appreciating?
>
> The precedence concept exists in egg as well:

The main concern I have at the moment is not so much that there is a
precendence model, or how much it can be tweaked (although I guess
that would be nice). It's more that as things stand, --use-wheel seems
essentially useless (if we assume that 99.999% of things you pip
install come ultimately from PyPI) unless you specify --no-index.

Paul.

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-24 @ 17:19
Talking about the design side of things separately...

On 24 October 2012 15:04, Daniel Holth <dholth@gmail.com> wrote:
> I would like to see the wheel ranking scheme extended to include any
> kind of dist. Imagine a cleanly sortable Dist() or LinkToDist() (bad
> names) that includes all kinds of dists including wheels for choosing
> the best from a particular version.

As a general concept, maybe this would be useful (although I don't
know in practice if tools would use this rather than inventing their
own scheme). But given that pip *only* installs wheels or sdists
(checkouts and develop dists are only ever specified explicitly, so
they never participate in prioritisation calculations), it's overkill
for pip.

> I think people will be very happy if there is a flag --prefer-wheel /
> --prefer-sdist. The extended ranking used to sort dists having the
> same version would be, depending on the flag,
>
> sdist
> other-kinds-of-dist?
> cp33-none-win32
> py3-none-any
>
> or
>
> cp33-none-win32
> py3-none-any
> sdist
> other-kinds-of-dist?

Ignoring other-kinds-of-dist as YAGNI, that's sdist-then-wheel or the
other way round. I'd argue that --use-wheel pretty much implies you
want to prefer wheels. When would you ever do --use-wheel
--prefer-sdist rather than just not considering wheels at all?

So we have

  --use-wheel => if there's a wheel use it, otherwise build from sdist
  (nothing) => build from sdist

No way of saying only use wheels if there's no sdist, but I don't see
a reason you'd need that (YAGNI again).

> Can there be a --prefer-local-wheel-remote-sdist without confusing everyone?

I can see the potential for wanting this (don't run binary code
downloaded off the internet) but if you actually care about that use
case, the following considerations apply:

1. Until there are actually significant numbers of wheels available on
PyPI this is irrelevant.
2. Signing is a better solution here (only use remote wheels if they
can be verified correct, otherwise use sdists) and even that hasn't
seen much interest yet.

Once again I'd say YAGNI.

<blue-sky>
Actually, this makes me think that a more useful way forward would be
setting up an infrastructure parallel to PyPI where people other than
project owners can upload wheels. So, for example, rather than having
my own local wheels for 40+ packages, and everyone else doing the
same, I can upload them to PyPI-binaries for general use (without
needing to submit them to the project owner who may not be interested
in wheels, etc, etc). There's a serious trust issue to consider here,
of course, but that might do a lot to encourage wider use of wheels.
People can always set up their own public hosting, of course, but a
central infrastructure would add credibility.

The key thing is the separation of project developers (who write the
code), and project builders (who package it).
</blue-sky>

Paul

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Daniel Holth
Date:
2012-10-24 @ 17:32
On Wed, Oct 24, 2012 at 1:19 PM, Paul Moore <p.f.moore@gmail.com> wrote:
> Talking about the design side of things separately...
>
> On 24 October 2012 15:04, Daniel Holth <dholth@gmail.com> wrote:
>> I would like to see the wheel ranking scheme extended to include any
>> kind of dist. Imagine a cleanly sortable Dist() or LinkToDist() (bad
>> names) that includes all kinds of dists including wheels for choosing
>> the best from a particular version.
>
> As a general concept, maybe this would be useful (although I don't
> know in practice if tools would use this rather than inventing their
> own scheme). But given that pip *only* installs wheels or sdists
> (checkouts and develop dists are only ever specified explicitly, so
> they never participate in prioritisation calculations), it's overkill
> for pip.
>
>> I think people will be very happy if there is a flag --prefer-wheel /
>> --prefer-sdist. The extended ranking used to sort dists having the
>> same version would be, depending on the flag,
>>
>> sdist
>> other-kinds-of-dist?
>> cp33-none-win32
>> py3-none-any
>>
>> or
>>
>> cp33-none-win32
>> py3-none-any
>> sdist
>> other-kinds-of-dist?
>
> Ignoring other-kinds-of-dist as YAGNI, that's sdist-then-wheel or the
> other way round. I'd argue that --use-wheel pretty much implies you
> want to prefer wheels. When would you ever do --use-wheel
> --prefer-sdist rather than just not considering wheels at all?

It's because someone might only upload the wheel to pypi. There might
not be an sdist.

> So we have
>
>   --use-wheel => if there's a wheel use it, otherwise build from sdist
>   (nothing) => build from sdist
>
> No way of saying only use wheels if there's no sdist, but I don't see
> a reason you'd need that (YAGNI again).
>
>> Can there be a --prefer-local-wheel-remote-sdist without confusing everyone?
>
> I can see the potential for wanting this (don't run binary code
> downloaded off the internet) but if you actually care about that use
> case, the following considerations apply:
>
> 1. Until there are actually significant numbers of wheels available on
> PyPI this is irrelevant.
> 2. Signing is a better solution here (only use remote wheels if they
> can be verified correct, otherwise use sdists) and even that hasn't
> seen much interest yet.
>
> Once again I'd say YAGNI.

Agreed. Too complicated, not very useful.

> <blue-sky>
> Actually, this makes me think that a more useful way forward would be
> setting up an infrastructure parallel to PyPI where people other than
> project owners can upload wheels. So, for example, rather than having
> my own local wheels for 40+ packages, and everyone else doing the
> same, I can upload them to PyPI-binaries for general use (without
> needing to submit them to the project owner who may not be interested
> in wheels, etc, etc). There's a serious trust issue to consider here,
> of course, but that might do a lot to encourage wider use of wheels.
> People can always set up their own public hosting, of course, but a
> central infrastructure would add credibility.
>
> The key thing is the separation of project developers (who write the
> code), and project builders (who package it).
> </blue-sky>

This exactly what ActiveState's pypm, enthought's enstaller, Debian,
RedHat, ... practically everyone uses binary packages this way. We
need to get better about including always including the full license
in the packages. "The above copyright notice and this permission
notice shall be included in all copies or substantial portions of the
Software."

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Marcus Smith
Date:
2012-10-24 @ 17:44
> Actually, this makes me think that a more useful way forward would be
> setting up an infrastructure parallel to PyPI where people other than
> project owners can upload wheels. So, for example, rather than having
> my own local wheels for 40+ packages, and everyone else doing the
> same, I can upload them to PyPI-binaries for general use (without
> needing to submit them to the project owner who may not be interested
> in wheels, etc, etc).


I was thinking of this too...

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Paul Moore
Date:
2012-10-24 @ 17:55
On 24 October 2012 18:44, Marcus Smith <qwcode@gmail.com> wrote:
>
>>
>> Actually, this makes me think that a more useful way forward would be
>> setting up an infrastructure parallel to PyPI where people other than
>> project owners can upload wheels. So, for example, rather than having
>> my own local wheels for 40+ packages, and everyone else doing the
>> same, I can upload them to PyPI-binaries for general use (without
>> needing to submit them to the project owner who may not be interested
>> in wheels, etc, etc).
>
>
> I was thinking of this too...

I'm not sure where would be a sensible place to discuss this. Is it
the sort of thing catalog-sig covers? (Oh, goody, another mailing list
to join :-))

Paul

Re: [wheel] Priorities (of source, wheel, local and downloaded)

From:
Daniel Holth
Date:
2012-10-24 @ 17:57
> I'm not sure where would be a sensible place to discuss this. Is it
> the sort of thing catalog-sig covers? (Oh, goody, another mailing list
> to join :-))

Donald Stufft (behind http://crate.io/) expressed interest in this idea.