Discussion:
[ITK-users] ITK Python: numpy to itk image (and viceversa)
D'Isidoro Fabio
2017-04-02 21:17:50 UTC
Permalink
Hallo,

I use ITK with Python Wrap. I need to interface my Python code with a Cython-wrapped C++ code that takes only numpy array as input and returns numpy array as output.

Hence, I need to convert the Python itk images into numpy array to be given as input to the wrapped C++ code, and then convert the numpy array in output from the wrapped C++ code back into python itk images.

Question 1) How can I do that in an efficient way? I found some posts on itk.PyBuffer but I could not find anywhere any reference on how to install it on my itk wrap build.

Question 2) The purpose of writing a part of my algorithm in C++ is to speed up the code. If the conversion between python itk images and numpy arrays is slow, I would lose all the speed gain obtained with the C++ implementation. Are there better ways to deal with that?

Thank you,

Fabio.
Matt McCormick
2017-04-03 18:40:50 UTC
Permalink
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and returns
numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is
a good way to build Cython-wrapped C++ code. We are using it for the
ITK and SimpleITK Python packages, and it has good Cython and CMake
support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be given
as input to the wrapped C++ code, and then convert the numpy array in output
from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts on
itk.PyBuffer but I could not find anywhere any reference on how to install
it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
additional documentation:

https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18

This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting

Module_BridgeNumPy=ON

in ITK's CMake configuration.


Since ITK 4.11.0, it is easier to build since it does not require the
NumPy headers.


In current ITK Git master (to be 4.12.0) the module is enabled by default.


Nightly ITK Python packages for ITK Git master are now being built:

https://github.com/InsightSoftwareConsortium/ITKPythonPackage

macOS and Linux are available. Windows packages will be available over
the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to speed
up the code. If the conversion between python itk images and numpy arrays is
slow, I would lose all the speed gain obtained with the C++ implementation.
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does
not do any copies during the conversion, and it is very fast.


HTH,
Matt


[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-users
D'Isidoro Fabio
2017-04-12 18:16:10 UTC
Permalink
Thank you. I am trying to build ITK Wrap Python with Module_BridgeNumPy=ON with Visual Studio 2017.

I get the following type of errors:

12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.10.25017/include\xstring(1905,26): error G3F63BFAE: constexpr variable '_Memcpy_move_offset' must be initialized by a constant expression
12> static constexpr size_t _Memcpy_move_offset = offsetof(_Mydata_t, _Bx);
12> ^ ~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.10.25017/include\stdexcept:23:21: note: in instantiation of template class 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.10.25017/include\xstring:1905:48: note: cast that performs the conversions of a reinterpret_cast is not allowed in a constant expression
12> static constexpr size_t _Memcpy_move_offset = offsetof(_Mydata_t, _Bx);
12> ^
12>C:/Program Files (x86)/Windows Kits/10/Include/10.0.10586.0/ucrt\stddef.h:38:32: note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const volatile&>((((s*)0)->m)))


The build worked with Visual Studio 2015 instead.

Can I fix this issue with Visual Studio 2017 somehow?

Thank you.

----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 Zürich, Switzerland

-----Original Message-----
From: Matt McCormick [mailto:***@kitware.com]
Sent: Montag, 3. April 2017 20:41
To: D'Isidoro Fabio <***@ethz.ch>
Cc: insight-***@itk.org
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)

Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a good way to build Cython-wrapped C++ code. We are using it for the ITK and SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some additional documentation:

https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18

This has been available in ITK for a few releases as a Remote module, which can be enabled by setting

Module_BridgeNumPy=ON

in ITK's CMake configuration.


Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.


In current ITK Git master (to be 4.12.0) the module is enabled by default.


Nightly ITK Python packages for ITK Git master are now being built:

https://github.com/InsightSoftwareConsortium/ITKPythonPackage

macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the C++ implementation.
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not do any copies during the conversion, and it is very fast.


HTH,
Matt


[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public
Dženan Zukić
2017-04-12 19:54:09 UTC
Permalink
Hi Fabio,

let me give it a try with VS2017 and Wrapping+NumPy. I will report back
when I have an update.

Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Post by D'Isidoro Fabio
Thank you. I am trying to build ITK Wrap Python with Module_BridgeNumPy=ON
with Visual Studio 2017.
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
ols/MSVC/14.10.25017/include\xstring(1905,26): error G3F63BFAE: constexpr
variable '_Memcpy_move_offset' must be initialized by a constant expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
ols/MSVC/14.10.25017/include\stdexcept:23:21: note: in instantiation of
template class 'std::basic_string<char, std::char_traits<char>,
std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
ols/MSVC/14.10.25017/include\xstring:1905:48: note: cast that performs
the conversions of a reinterpret_cast is not allowed in a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const
volatile&>((((s*)0)->m)))
The build worked with Visual Studio 2015 instead.
Can I fix this issue with Visual Studio 2017 somehow?
Thank you.
----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 ZÃŒrich, Switzerland
-----Original Message-----
Sent: Montag, 3. April 2017 20:41
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a
good way to build Cython-wrapped C++ code. We are using it for the ITK and
SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18
This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting
Module_BridgeNumPy=ON
in ITK's CMake configuration.
Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.
In current ITK Git master (to be 4.12.0) the module is enabled by default.
https://github.com/InsightSoftwareConsortium/ITKPythonPackage
macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the
C++ implementation.
Post by D'Isidoro Fabio
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not
do any copies during the conversion, and it is very fast.
HTH,
Matt
[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
D'Isidoro Fabio
2017-04-12 20:02:58 UTC
Permalink
Ok, thanks. Moe specifically I am using now Visual Studio 2017 Preview.

Fabio.
From: DÅŸenan Zukić [mailto:***@gmail.com]
Sent: Mittwoch, 12. April 2017 21:54
To: D'Isidoro Fabio <***@ethz.ch>
Cc: Matt McCormick <***@kitware.com>; insight-***@itk.org
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)

Hi Fabio,

let me give it a try with VS2017 and Wrapping+NumPy. I will report back when I have an update.

Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)

On Wed, Apr 12, 2017 at 2:16 PM, D'Isidoro Fabio <***@ethz.ch<mailto:***@ethz.ch>> wrote:
Thank you. I am trying to build ITK Wrap Python with Module_BridgeNumPy=ON with Visual Studio 2017.

I get the following type of errors:

12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.10.25017/include\xstring(1905,26): error G3F63BFAE: constexpr variable '_Memcpy_move_offset' must be initialized by a constant expression
12> static constexpr size_t _Memcpy_move_offset = offsetof(_Mydata_t, _Bx);
12> ^ ~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.10.25017/include\stdexcept:23:21: note: in instantiation of template class 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/Tools/MSVC/14.10.25017/include\xstring:1905:48: note: cast that performs the conversions of a reinterpret_cast is not allowed in a constant expression
12> static constexpr size_t _Memcpy_move_offset = offsetof(_Mydata_t, _Bx);
12> ^
12>C:/Program Files (x86)/Windows Kits/10/Include/10.0.10586.0/ucrt\stddef.h:38:32: note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const volatile&>((((s*)0)->m)))


The build worked with Visual Studio 2015 instead.

Can I fix this issue with Visual Studio 2017 somehow?

Thank you.

----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 ZÃŒrich, Switzerland

-----Original Message-----
From: Matt McCormick [mailto:***@kitware.com<mailto:***@kitware.com>]
Sent: Montag, 3. April 2017 20:41
To: D'Isidoro Fabio <***@ethz.ch<mailto:***@ethz.ch>>
Cc: insight-***@itk.org<mailto:insight-***@itk.org>
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)

Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a good way to build Cython-wrapped C++ code. We are using it for the ITK and SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some additional documentation:

https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18

This has been available in ITK for a few releases as a Remote module, which can be enabled by setting

Module_BridgeNumPy=ON

in ITK's CMake configuration.


Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.


In current ITK Git master (to be 4.12.0) the module is enabled by default.


Nightly ITK Python packages for ITK Git master are now being built:

https://github.com/InsightSoftwareConsortium/ITKPythonPackage

macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the C++ implementation.
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not do any copies during the conversion, and it is very fast.


HTH,
Matt


[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-users
Dženan Zukić
2017-04-12 20:26:06 UTC
Permalink
I reproduced the error. I am looking into fixing it.
Ok, thanks. Moe specifically I am using now *Visual Studio 2017 Preview*.
Fabio.
*Sent:* Mittwoch, 12. April 2017 21:54
*Subject:* Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hi Fabio,
let me give it a try with VS2017 and Wrapping+NumPy. I will report back
when I have an update.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Thank you. I am trying to build ITK Wrap Python with Module_BridgeNumPy=ON
with Visual Studio 2017.
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/
constexpr variable '_Memcpy_move_offset' must be initialized by a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/
Tools/MSVC/14.10.25017/include\stdexcept:23:21: note: in instantiation of
template class 'std::basic_string<char, std::char_traits<char>,
std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/
Tools/MSVC/14.10.25017/include\xstring:1905:48: note: cast that performs
the conversions of a reinterpret_cast is not allowed in a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const
volatile&>((((s*)0)->m)))
The build worked with Visual Studio 2015 instead.
Can I fix this issue with Visual Studio 2017 somehow?
Thank you.
----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 ZÃŒrich, Switzerland
-----Original Message-----
Sent: Montag, 3. April 2017 20:41
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a
good way to build Cython-wrapped C++ code. We are using it for the ITK and
SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18
This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting
Module_BridgeNumPy=ON
in ITK's CMake configuration.
Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.
In current ITK Git master (to be 4.12.0) the module is enabled by default.
https://github.com/InsightSoftwareConsortium/ITKPythonPackage
macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the
C++ implementation.
Post by D'Isidoro Fabio
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not
do any copies during the conversion, and it is very fast.
HTH,
Matt
[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
Francois Budin
2017-04-12 20:26:34 UTC
Permalink
Fabio,

To add a little bit of extra information about using BridgeNumpy:

From NumPy to ITK, the memory is never copied, and always shared between
the NumPy array and the ITK image.
From ITK to NumPy, the memory is not copied at first, if the data is not
changed from the NumPy side (e.g. if the ITK image is changed, the NumPy
array will be modified too), but if you modify the NumPy array directly,
NumPy will create a copy of the data. This also means that if you were to
modify the NumPy array, the modifications would not be visible in the ITK
image.

A limited workaround to this is:

# Get NumPy array from image
arr_image = itk.GetArrayFromImage(image)
# Create copy of array that can be modified in-place
arr = arr_image.copy()
# Modify array


# Update image in-place with new array
arr_image.setfield(arr,arr_image.dtype)

The advantage of this method is that the image will be updated. However,
there is an explicit copy of your data.
If the copy step is skipped and the original NumPy array is modified, a
copy of the data will be performed by NumPy and you may not be aware of it
until you see that the data in the NumPy array and the image do not match
anymore.

Hope this helps,
Francois
Ok, thanks. Moe specifically I am using now *Visual Studio 2017 Preview*.
Fabio.
*Sent:* Mittwoch, 12. April 2017 21:54
*Subject:* Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hi Fabio,
let me give it a try with VS2017 and Wrapping+NumPy. I will report back
when I have an update.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Thank you. I am trying to build ITK Wrap Python with Module_BridgeNumPy=ON
with Visual Studio 2017.
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/
constexpr variable '_Memcpy_move_offset' must be initialized by a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/
Tools/MSVC/14.10.25017/include\stdexcept:23:21: note: in instantiation of
template class 'std::basic_string<char, std::char_traits<char>,
std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/
Tools/MSVC/14.10.25017/include\xstring:1905:48: note: cast that performs
the conversions of a reinterpret_cast is not allowed in a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const
volatile&>((((s*)0)->m)))
The build worked with Visual Studio 2015 instead.
Can I fix this issue with Visual Studio 2017 somehow?
Thank you.
----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 ZÃŒrich, Switzerland
-----Original Message-----
Sent: Montag, 3. April 2017 20:41
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a
good way to build Cython-wrapped C++ code. We are using it for the ITK and
SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18
This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting
Module_BridgeNumPy=ON
in ITK's CMake configuration.
Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.
In current ITK Git master (to be 4.12.0) the module is enabled by default.
https://github.com/InsightSoftwareConsortium/ITKPythonPackage
macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the
C++ implementation.
Post by D'Isidoro Fabio
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not
do any copies during the conversion, and it is very fast.
HTH,
Matt
[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
Dženan Zukić
2017-04-12 20:50:08 UTC
Permalink
Hi Fabio,

it looks like we will have to wait somewhat until VS writers get around to
implement __builtin_offsetof
<https://developercommunity.visualstudio.com/content/problem/22196/static-assert-cannot-compile-constexprs-method-tha.html>

I don't know why it doesn't trigger when Python wrapping is OFF. I will
look into it more.

Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Post by Matt McCormick
Fabio,
From NumPy to ITK, the memory is never copied, and always shared between
the NumPy array and the ITK image.
From ITK to NumPy, the memory is not copied at first, if the data is not
changed from the NumPy side (e.g. if the ITK image is changed, the NumPy
array will be modified too), but if you modify the NumPy array directly,
NumPy will create a copy of the data. This also means that if you were to
modify the NumPy array, the modifications would not be visible in the ITK
image.
# Get NumPy array from image
arr_image = itk.GetArrayFromImage(image)
# Create copy of array that can be modified in-place
arr = arr_image.copy()
# Modify array


# Update image in-place with new array
arr_image.setfield(arr,arr_image.dtype)
The advantage of this method is that the image will be updated. However,
there is an explicit copy of your data.
If the copy step is skipped and the original NumPy array is modified, a
copy of the data will be performed by NumPy and you may not be aware of it
until you see that the data in the NumPy array and the image do not match
anymore.
Hope this helps,
Francois
Ok, thanks. Moe specifically I am using now *Visual Studio 2017 Preview*.
Fabio.
*Sent:* Mittwoch, 12. April 2017 21:54
*Subject:* Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hi Fabio,
let me give it a try with VS2017 and Wrapping+NumPy. I will report back
when I have an update.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Thank you. I am trying to build ITK Wrap Python with
Module_BridgeNumPy=ON with Visual Studio 2017.
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
constexpr variable '_Memcpy_move_offset' must be initialized by a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
ols/MSVC/14.10.25017/include\stdexcept:23:21: note: in instantiation of
template class 'std::basic_string<char, std::char_traits<char>,
std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
ols/MSVC/14.10.25017/include\xstring:1905:48: note: cast that performs
the conversions of a reinterpret_cast is not allowed in a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const
volatile&>((((s*)0)->m)))
The build worked with Visual Studio 2015 instead.
Can I fix this issue with Visual Studio 2017 somehow?
Thank you.
----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 ZÃŒrich, Switzerland
-----Original Message-----
Sent: Montag, 3. April 2017 20:41
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a
good way to build Cython-wrapped C++ code. We are using it for the ITK and
SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18
This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting
Module_BridgeNumPy=ON
in ITK's CMake configuration.
Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.
In current ITK Git master (to be 4.12.0) the module is enabled by default.
https://github.com/InsightSoftwareConsortium/ITKPythonPackage
macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the
C++ implementation.
Post by D'Isidoro Fabio
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not
do any copies during the conversion, and it is very fast.
HTH,
Matt
[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
Dženan Zukić
2017-04-13 14:48:40 UTC
Permalink
Hi Fabio,

the first error is coming from ITKCommonCastXML project, and the other
first 1000 errors are also coming from*CastXML and *Swig projects. This
means that the wrapping infrastructure is somehow triggering that bug in
VS2017.

The easiest thing to try is update CastXML, which Matt said would take care
of. Let's see if that helps.

Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Post by Dženan Zukić
Hi Fabio,
it looks like we will have to wait somewhat until VS writers get around to
implement __builtin_offsetof
<https://developercommunity.visualstudio.com/content/problem/22196/static-assert-cannot-compile-constexprs-method-tha.html>
I don't know why it doesn't trigger when Python wrapping is OFF. I will
look into it more.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
On Wed, Apr 12, 2017 at 4:26 PM, Francois Budin <
Post by Matt McCormick
Fabio,
From NumPy to ITK, the memory is never copied, and always shared between
the NumPy array and the ITK image.
From ITK to NumPy, the memory is not copied at first, if the data is not
changed from the NumPy side (e.g. if the ITK image is changed, the NumPy
array will be modified too), but if you modify the NumPy array directly,
NumPy will create a copy of the data. This also means that if you were to
modify the NumPy array, the modifications would not be visible in the ITK
image.
# Get NumPy array from image
arr_image = itk.GetArrayFromImage(image)
# Create copy of array that can be modified in-place
arr = arr_image.copy()
# Modify array


# Update image in-place with new array
arr_image.setfield(arr,arr_image.dtype)
The advantage of this method is that the image will be updated. However,
there is an explicit copy of your data.
If the copy step is skipped and the original NumPy array is modified, a
copy of the data will be performed by NumPy and you may not be aware of it
until you see that the data in the NumPy array and the image do not match
anymore.
Hope this helps,
Francois
Ok, thanks. Moe specifically I am using now *Visual Studio 2017 Preview*
.
Fabio.
*Sent:* Mittwoch, 12. April 2017 21:54
*Subject:* Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hi Fabio,
let me give it a try with VS2017 and Wrapping+NumPy. I will report back
when I have an update.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Thank you. I am trying to build ITK Wrap Python with
Module_BridgeNumPy=ON with Visual Studio 2017.
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
constexpr variable '_Memcpy_move_offset' must be initialized by a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
ols/MSVC/14.10.25017/include\stdexcept:23:21: note: in instantiation of
template class 'std::basic_string<char, std::char_traits<char>,
std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual Studio/Preview/Community/VC/To
ols/MSVC/14.10.25017/include\xstring:1905:48: note: cast that performs
the conversions of a reinterpret_cast is not allowed in a constant
expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const
volatile&>((((s*)0)->m)))
The build worked with Visual Studio 2015 instead.
Can I fix this issue with Visual Studio 2017 somehow?
Thank you.
----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 ZÃŒrich, Switzerland
-----Original Message-----
Sent: Montag, 3. April 2017 20:41
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a
good way to build Cython-wrapped C++ code. We are using it for the ITK and
SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18
This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting
Module_BridgeNumPy=ON
in ITK's CMake configuration.
Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.
In current ITK Git master (to be 4.12.0) the module is enabled by default.
https://github.com/InsightSoftwareConsortium/ITKPythonPackage
macOS and Linux are available. Windows packages will be available over
the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with
the C++ implementation.
Post by D'Isidoro Fabio
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not
do any copies during the conversion, and it is very fast.
HTH,
Matt
[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
Dženan Zukić
2017-04-19 19:57:09 UTC
Permalink
Hi Fabio,

I just tested the master branch after CastXML was updated. Unfortunately,
it didn't help. The compile errors are still present in VS2017 with
wrapping.

Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Post by Dženan Zukić
Hi Fabio,
the first error is coming from ITKCommonCastXML project, and the other
first 1000 errors are also coming from*CastXML and *Swig projects. This
means that the wrapping infrastructure is somehow triggering that bug in
VS2017.
The easiest thing to try is update CastXML, which Matt said would take
care of. Let's see if that helps.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Post by Dženan Zukić
Hi Fabio,
it looks like we will have to wait somewhat until VS writers get around
to implement __builtin_offsetof
<https://developercommunity.visualstudio.com/content/problem/22196/static-assert-cannot-compile-constexprs-method-tha.html>
I don't know why it doesn't trigger when Python wrapping is OFF. I will
look into it more.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
On Wed, Apr 12, 2017 at 4:26 PM, Francois Budin <
Post by Matt McCormick
Fabio,
From NumPy to ITK, the memory is never copied, and always shared between
the NumPy array and the ITK image.
From ITK to NumPy, the memory is not copied at first, if the data is not
changed from the NumPy side (e.g. if the ITK image is changed, the NumPy
array will be modified too), but if you modify the NumPy array directly,
NumPy will create a copy of the data. This also means that if you were to
modify the NumPy array, the modifications would not be visible in the ITK
image.
# Get NumPy array from image
arr_image = itk.GetArrayFromImage(image)
# Create copy of array that can be modified in-place
arr = arr_image.copy()
# Modify array


# Update image in-place with new array
arr_image.setfield(arr,arr_image.dtype)
The advantage of this method is that the image will be updated. However,
there is an explicit copy of your data.
If the copy step is skipped and the original NumPy array is modified, a
copy of the data will be performed by NumPy and you may not be aware of it
until you see that the data in the NumPy array and the image do not match
anymore.
Hope this helps,
Francois
Ok, thanks. Moe specifically I am using now *Visual Studio 2017 Preview*.
Fabio.
*Sent:* Mittwoch, 12. April 2017 21:54
*Subject:* Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hi Fabio,
let me give it a try with VS2017 and Wrapping+NumPy. I will report back
when I have an update.
Regards,
DÅŸenan Zukić, PhD, Senior R&D Engineer, Kitware (Carrboro, N.C.)
Thank you. I am trying to build ITK Wrap Python with
Module_BridgeNumPy=ON with Visual Studio 2017.
12>C:/Program Files (x86)/Microsoft Visual
error G3F63BFAE: constexpr variable '_Memcpy_move_offset' must be
initialized by a constant expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
~~~~~~~~~~~~~~~~~~~~~~~~
12>C:/Program Files (x86)/Microsoft Visual
note: in instantiation of template class 'std::basic_string<char,
std::char_traits<char>, std::allocator<char> >' requested here
12> : _Mybase(_Message.c_str())
12> ^
12>C:/Program Files (x86)/Microsoft Visual
note: cast that performs the conversions of a reinterpret_cast is not
allowed in a constant expression
12> static constexpr size_t _Memcpy_move_offset =
offsetof(_Mydata_t, _Bx);
12> ^
note: expanded from macro 'offsetof'
12> #define offsetof(s,m) ((size_t)&reinterpret_cast<char const
volatile&>((((s*)0)->m)))
The build worked with Visual Studio 2015 instead.
Can I fix this issue with Visual Studio 2017 somehow?
Thank you.
----------------------------------------------------------------------
Fabio D’Isidoro - PhD Student
Institute of Biomechanics
HPP O 14
Hönggerbergring 64
8093 ZÃŒrich, Switzerland
-----Original Message-----
Sent: Montag, 3. April 2017 20:41
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a
good way to build Cython-wrapped C++ code. We are using it for the ITK and
SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18
This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting
Module_BridgeNumPy=ON
in ITK's CMake configuration.
Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.
In current ITK Git master (to be 4.12.0) the module is enabled by default.
https://github.com/InsightSoftwareConsortium/ITKPythonPackage
macOS and Linux are available. Windows packages will be available over
the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with
the C++ implementation.
Post by D'Isidoro Fabio
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does
not do any copies during the conversion, and it is very fast.
HTH,
Matt
[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
_____________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
http://www.kitware.com/products/protraining.php
http://www.itk.org/Wiki/ITK_FAQ
http://public.kitware.com/mailman/listinfo/insight-users
D'Isidoro Fabio
2017-04-22 15:48:44 UTC
Permalink
Thank you Matt,

I am looking now at scikit-build.

Do you think I can use it to build C++/CUDA accelerated codes that use ITK too?

Is there a link to a simple example of how to setup a scikit-build for an ITK-based C++ code?

Thank you,
Fabio.

-----Original Message-----
From: Matt McCormick [mailto:***@kitware.com]
Sent: Montag, 3. April 2017 20:41
To: D'Isidoro Fabio <***@ethz.ch>
Cc: insight-***@itk.org
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)

Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a good way to build Cython-wrapped C++ code. We are using it for the ITK and SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some additional documentation:

https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18

This has been available in ITK for a few releases as a Remote module, which can be enabled by setting

Module_BridgeNumPy=ON

in ITK's CMake configuration.


Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.


In current ITK Git master (to be 4.12.0) the module is enabled by default.


Nightly ITK Python packages for ITK Git master are now being built:

https://github.com/InsightSoftwareConsortium/ITKPythonPackage

macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the C++ implementation.
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not do any copies during the conversion, and it is very fast.


HTH,
Matt


[1] http://scikit-build.org/
_____________________________________
Powered by www.kitware.com

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-users
Lowekamp, Bradley (NIH/NLM/LHC) [C]
2017-04-24 15:35:59 UTC
Permalink
Hello,

To the best of my knowledge and experience with the ITK pipeline, when the output of a filter ( H or R ) is used as multiple inputs with the ReleaseData, the data from the output ( H or R ) will be release after _each_ filter which use that output data. So the result is that the minimal amount of memory is consumed! But in a complex pipeline like yours filters will be executed multiple times for a single update.

Based on your comments you likely want to set only R and W to ReleaseData automatically. This ensures that all the inputs to M are cached and available. The trade off is that R will get execute twice. Alternatively, you could just manually release the output of R after execution.

HTH,
Brad

On Apr 22, 2017, at 12:46 PM, asertyuio via Insight-users <insight-***@itk.org<mailto:insight-***@itk.org>> wrote:


Dear Users,

I have trouble applying the concepts behind the release data flag, and memory management of itk to a particular problem.

I’m developing a filter with, for part of it, it a rather complex mini pipeline.
H, R, RS, W, WS and M are itk filters.
The filter will be updated frequently, and I’m trying to make the update only update necessary parts, while keeping memory imprinting as low as possible.

Here is a schematic, simplified graph representing my filter :

H
| \
v \
R \ -> take as input a parameter P1 that do not change often
| \
/ \ /\
v v v |
RS W |
| | |
| v |
\ WS |
\ | /
v v v
M -> takes parameters P2 that are often updated


I would like to be able to release the output data of the filter R and of the filter W, updating them only if the parameter P1 change (which would trigger update for the whole pipeline), while still being able to change P2 without having to recalculate R, W and RS.

I think it may be possible with the releaseData flag, don’t really understand where it should be putted, and if this will do what I need.

Thanks a lot for you help !
Yann

​
_____________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-users
Lowekamp, Bradley (NIH/NLM/LHC) [C]
2017-04-25 17:19:14 UTC
Permalink
Hello,

You also need to graft the input and output so that updates and propagation of the mini-pipeline does not propagate beyond the mini-pipeline[1] [2].

Your approach sounds about right. But there are a lot of stumbling block with this approach and just a couple ways to do everything “proper”. Also this goal of “caching” the internal data, is not what is generally done in ITK. Most filters in ITK, prefer to release the data and keep memory usage to a minimum. It frequently turns out, that keeping a pipeline’s memory foot print small is important to fast execution and allows the processing to scale to larger data sets too.

The “ReleaseData” flag does not cause the pipeline to be re-executed. You will have to examine the pipeline code and the time stamps of the relevant objects to determine the effect of “manual” releasing the data, and how to best do it for your situation.

HTH,
Brad

[1] https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Filtering/ImageFeature/include/itkUnsharpMaskImageFilter.hxx#L78-L79
[2] https://github.com/InsightSoftwareConsortium/ITK/blob/master/Modules/Filtering/ImageFeature/include/itkUnsharpMaskImageFilter.hxx#L100-L102

On Apr 25, 2017, at 11:55 AM, asertyuio via Insight-users <insight-***@itk.org<mailto:insight-***@itk.org>> wrote:


Thanks for your answer Brad !

Just to confirm that I understood well:

To unsure that only the needed parts are actualized I call the global update, I need to:

*
use the H, R, RS, W, WS, M filters as members
*
set up their relationships in the filter constructor
*
do the update at the M filter

Then I can call manually releaseData on the R and W filter after the call to M update().

And that should do the trick, and release data should not make the downstream filters RS and WS to update, right ?

Ok, after rereading what the software guide tells about that:

Filter execution occurs
when a) the filter is modified as a result of modifying an instance variable; b) the input to the filter
changes; c) the input data has been released
— ITK Software guide p. 200

So it seems like if I release the data of W or R, updating M will force WS and RS to update
 Am I wrong ?

If I want the filter to have the expected behavior, Is it possible to trigger the update manually ?
I’m thinking of disconnecting RS and WS by using new smart pointers for R and W output, checking manually the modified state to conditionally call update when relevant.

Yann

Le 24/04/2017 à 17:35, Lowekamp, Bradley (NIH/NLM/LHC) [C] a écrit :


Hello,

To the best of my knowledge and experience with the ITK pipeline, when the output of a filter ( H or R ) is used as multiple inputs with the ReleaseData, the data from the output ( H or R ) will be release after _each_ filter which use that output data. So the result is that the minimal amount of memory is consumed! But in a complex pipeline like yours filters will be executed multiple times for a single update.

Based on your comments you likely want to set only R and W to ReleaseData automatically. This ensures that all the inputs to M are cached and available. The trade off is that R will get execute twice. Alternatively, you could just manually release the output of R after execution.

HTH,
Brad

On Apr 22, 2017, at 12:46 PM, asertyuio via Insight-users <insight-***@itk.org<mailto:insight-***@itk.org>> wrote:


Dear Users,

I have trouble applying the concepts behind the release data flag, and memory management of itk to a particular problem.

I’m developing a filter with, for part of it, it a rather complex mini pipeline.
H, R, RS, W, WS and M are itk filters.
The filter will be updated frequently, and I’m trying to make the update only update necessary parts, while keeping memory imprinting as low as possible.

Here is a schematic, simplified graph representing my filter :

H
| \
v \
R \ -> take as input a parameter P1 that do not change often
| \
/ \ /\
v v v |
RS W |
| | |
| v |
\ WS |
\ | /
v v v
M -> takes parameters P2 that are often updated


I would like to be able to release the output data of the filter R and of the filter W, updating them only if the parameter P1 change (which would trigger update for the whole pipeline), while still being able to change P2 without having to recalculate R, W and RS.

I think it may be possible with the releaseData flag, don’t really understand where it should be putted, and if this will do what I need.

Thanks a lot for you help !
Yann

​
_____________________________________
Powered by www.kitware.com<http://www.kitware.com/>

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-users



​
_____________________________________
Powered by www.kitware.com<http://www.kitware.com>

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Kitware offers ITK Training Courses, for more information visit:
http://www.kitware.com/products/protraining.php

Please keep messages on-topic and check the ITK FAQ at:
http://www.itk.org/Wiki/ITK_FAQ

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/insight-users

Matt McCormick
2017-04-24 17:39:51 UTC
Permalink
Hi Fabio,

Yes, building Python extensions that depend on libraries like Cuda is muich
easier with CMake build system support in scikit-build.

I will follow up with an example of how to set up scikit-build for
ITK-based C++ code.

Thanks,
Matt
Post by D'Isidoro Fabio
Thank you Matt,
I am looking now at scikit-build.
Do you think I can use it to build C++/CUDA accelerated codes that use ITK too?
Is there a link to a simple example of how to setup a scikit-build for an
ITK-based C++ code?
Thank you,
Fabio.
-----Original Message-----
Sent: Montag, 3. April 2017 20:41
Subject: Re: [ITK-users] ITK Python: numpy to itk image (and viceversa)
Hallo Fabio,
Post by D'Isidoro Fabio
I use ITK with Python Wrap. I need to interface my Python code with a
Cython-wrapped C++ code that takes only numpy array as input and
returns numpy array as output.
Cool. By the way, you may be interested in scikit-build [1], which is a
good way to build Cython-wrapped C++ code. We are using it for the ITK and
SimpleITK Python packages, and it has good Cython and CMake support.
Post by D'Isidoro Fabio
Hence, I need to convert the Python itk images into numpy array to be
given as input to the wrapped C++ code, and then convert the numpy
array in output from the wrapped C++ code back into python itk images.
Question 1) How can I do that in an efficient way? I found some posts
on itk.PyBuffer but I could not find anywhere any reference on how to
install it on my itk wrap build.
Yes, itk.PyBuffer works great for that. Please review a PR for some
https://github.com/InsightSoftwareConsortium/ITKBridgeNumPy/pull/18
This has been available in ITK for a few releases as a Remote module,
which can be enabled by setting
Module_BridgeNumPy=ON
in ITK's CMake configuration.
Since ITK 4.11.0, it is easier to build since it does not require the NumPy headers.
In current ITK Git master (to be 4.12.0) the module is enabled by default.
https://github.com/InsightSoftwareConsortium/ITKPythonPackage
macOS and Linux are available. Windows packages will be available over the coming weeks.
Post by D'Isidoro Fabio
Question 2) The purpose of writing a part of my algorithm in C++ is to
speed up the code. If the conversion between python itk images and
numpy arrays is slow, I would lose all the speed gain obtained with the
C++ implementation.
Post by D'Isidoro Fabio
Are there better ways to deal with that?
The newer versions ITKBridgeNumPy use a NumPy array view, which does not
do any copies during the conversion, and it is very fast.
HTH,
Matt
[1] http://scikit-build.org/
Loading...