test test_slice failed -- [9, 7, 5, 3, 1] == [0]

[Anders Qvist]

Between 1036491804 and 1036522446 (should be GMT 2002-11-05, 10:20 and
2002-11-05, 18:54 respectively), test_slice started failing with the
following message:

test test_slice failed -- [9, 7, 5, 3, 1] == [0]

om moghedien; linux/alpha. There has been some changes containing
pointer arithmetic around line 160 of sliceobjects.c, which might be
guilty (just a shot in the dark; 64-bit clean?):

+++ Objects/sliceobject.c 5 Nov 2002 15:28:51 -0000 2.19

- if ((*stop - *start)*(*step) <= 0) {
+ if ((*step < 0 && *stop >= *start)
+ || (*step > 0 && *start >= *stop)) {
*slicelength = 0;
}

I'll be back toworrow with the results of a manual test build.
--
Anders "Quest" Qvist

"We've all heard that a million monkeys banging on a million typewriters
will eventually reproduce the entire works of Shakespeare. Now, thanks
to the Internet, we know this is not true." -- Robert Wilensky

[Neal Norwitz]

Post by Anders Qvist
Between 1036491804 and 1036522446 (should be GMT 2002-11-05, 10:20 and
2002-11-05, 18:54 respectively), test_slice started failing with the
test test_slice failed -- [9, 7, 5, 3, 1] == [0]
om moghedien; linux/alpha. There has been some changes containing
pointer arithmetic around line 160 of sliceobjects.c, which might be

It is a 64-bit problem, but doesn't have to do with the checkin to
sliceobject. The new test (below) broke with old code too.

vereq(range(10)[::sys.maxint - 1], [0])

The problem is that sizeof(int) != sizeof(long). The parameters for
PySlice_GetIndices() and PySlice_GetIndicesEx() take ints, not longs.
So the code is doing:

range(10)[::-2]

since the high 32 bits are lost.

The only solution I can think of is to have 2 interfaces:
- the old API which takes int/int*, PySlice_GetIndices()
deprecate this API and raise overflow exception
if (int)value != (long)value
- the new API which takes long/long*, PySlice_GetIndicesEx()

PySlice_GetIndicesEx() was added in 2.3.

Otherwise, we are stuck with slices only being able to support
32 bits even on 64 bit architectures.

Other ideas?

Neal

[Guido van Rossum]

Post by Neal Norwitz
It is a 64-bit problem, but doesn't have to do with the checkin to
sliceobject. The new test (below) broke with old code too.
vereq(range(10)[::sys.maxint - 1], [0])
The problem is that sizeof(int) != sizeof(long). The parameters for
PySlice_GetIndices() and PySlice_GetIndicesEx() take ints, not longs.
range(10)[::-2]
since the high 32 bits are lost.
- the old API which takes int/int*, PySlice_GetIndices()
deprecate this API and raise overflow exception
if (int)value != (long)value
- the new API which takes long/long*, PySlice_GetIndicesEx()
PySlice_GetIndicesEx() was added in 2.3.

You meant "must be added" right?

Post by Neal Norwitz
Otherwise, we are stuck with slices only being able to support
32 bits even on 64 bit architectures.

But *everything* having to do with sequences only supports 32 bits:
ob_size, PySequence_Length() the arguments to PySequence_GetItem(),
etc.

Unless you want to fix all those (breaking backwards compatibility),
I'm not sure why you'd want to fix PySlice_GetIndices()...

--Guido van Rossum (home page: http://www.python.org/~guido/)

[Neal Norwitz]

Post by Guido van Rossum

Post by Neal Norwitz
PySlice_GetIndicesEx() was added in 2.3.

You meant "must be added" right?

No, this was an API Michael added when doing the extended slicing, AFAIK.

Post by Guido van Rossum

Post by Neal Norwitz
Otherwise, we are stuck with slices only being able to support
32 bits even on 64 bit architectures.

ob_size, PySequence_Length() the arguments to PySequence_GetItem(),
etc.
Unless you want to fix all those (breaking backwards compatibility),
I'm not sure why you'd want to fix PySlice_GetIndices()...

You're right. This gets me thinking about starting to remove
the 32 bit limitation...

I looked at how hard it would be to support 64 bit sequences.

If we did this, 64 bit users would be able to use long sequences at
the expense of backwards compatibility. Also, we would need something
like Py_seq_len, use a configure option, and all the ob_size,
arguments etc. would have to use the new type name.

I did a grep for int, there were a ton. But it looks like
those used for sequence lengths were much fewer.

I suppose this change could be accomplished in stages.

Step 1, typedef int Py_seq_len; for all platforms.
Step 2, start using it for the proper APIs in the header files.
Step 3, change the C files.
Repeat steps 2 & 3 until done.

I'm not volunteering, of course. :-) I don't need this and have no
system to use for testing. So this is all academic at best.

Neal

[Guido van Rossum]

Post by Neal Norwitz
I looked at how hard it would be to support 64 bit sequences.
If we did this, 64 bit users would be able to use long sequences at
the expense of backwards compatibility. Also, we would need something
like Py_seq_len, use a configure option, and all the ob_size,
arguments etc. would have to use the new type name.
I did a grep for int, there were a ton. But it looks like
those used for sequence lengths were much fewer.
I suppose this change could be accomplished in stages.
Step 1, typedef int Py_seq_len; for all platforms.
Step 2, start using it for the proper APIs in the header files.
Step 3, change the C files.
Repeat steps 2 & 3 until done.
I'm not volunteering, of course. :-) I don't need this and have no
system to use for testing. So this is all academic at best.

I have no time to do it either, but I hereby officially pronounce that
I wouldn't mind breaking binary compatibility for this, once, at 2.3
or 2.4. Maybe someone gets inspired.

The proper type should probably be size_t or perhaps ssize_t, BTW.

Also, Python ints should use an integral type that's at least as wide
as a pointer; at least on Windows64, long is 32 bits but pointers are
64 bits, so long isn't it. If Python ints could be 64 bits even on 32
bit platforms (if long long is supported) that would be cool to.
It could possibly break more stuff if sys.maxint doesn't fit
in a C long -- but that would already be a problem for Win64.

--Guido van Rossum (home page: http://www.python.org/~guido/)