NumericArray versus PackedArray in MMA12 Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) Announcing the arrival of Valued Associate #679: Cesar Manara Unicorn Meta Zoo #1: Why another podcast?How can I work with RawArray?How to convert a column of a Dataset into a list
Why do people think Winterfell crypts is the safest place for women, children & old people?
Why not use the yoke to control yaw, as well as pitch and roll?
Married in secret, can marital status in passport be changed at a later date?
How do I deal with an erroneously large refund?
Protagonist's race is hidden - should I reveal it?
Suing a Police Officer Instead of the Police Department
What came first? Venom as the movie or as the song?
Providing direct feedback to a product salesperson
How to create a command for the "strange m" symbol in latex?
How can I introduce the names of fantasy creatures to the reader?
Proving inequality for positive definite matrix
Who's this lady in the war room?
Converting a text document with special format to Pandas DataFrame
How to leave only the following strings?
Marquee sign letters
Is it OK if I do not take the receipt in Germany?
Is "ein Herz wie das meine" an antiquated or colloquial use of the possesive pronoun?
How was Lagrange appointed professor of mathematics so early?
lm and glm function in R
Can a Wizard take the Magic Initiate feat and select spells from the Wizard list?
Is the Mordenkainen's Sword spell underpowered?
Unix AIX passing variable and arguments to expect and spawn
Weaponising the Grasp-at-a-Distance spell
2 sample t test for sample sizes - 30,000 and 150,000
NumericArray versus PackedArray in MMA12
Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)
Announcing the arrival of Valued Associate #679: Cesar Manara
Unicorn Meta Zoo #1: Why another podcast?How can I work with RawArray?How to convert a column of a Dataset into a list
$begingroup$
I just tried to understand the differences between NumericArray and PackedArray in MMA12. The memory footprint of a NumericArray appears to be less than for a PackedArray. However, what is the use of the NumericArray, if simple matrix multiplication of NumericArrays
A.B
returns unevaluated? In fact, Flatten[] works with NumericArray[] but Dot[] does not.
So, can someone explain to me the purpose and/or use of NumericArray[]?
data-types
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
$endgroup$
add a comment |
$begingroup$
I just tried to understand the differences between NumericArray and PackedArray in MMA12. The memory footprint of a NumericArray appears to be less than for a PackedArray. However, what is the use of the NumericArray, if simple matrix multiplication of NumericArrays
A.B
returns unevaluated? In fact, Flatten[] works with NumericArray[] but Dot[] does not.
So, can someone explain to me the purpose and/or use of NumericArray[]?
data-types
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
$endgroup$
$begingroup$
Some details here: chat.stackexchange.com/transcript/message/49966830#49966830
$endgroup$
– Carl Lange
Apr 18 at 9:23
2
$begingroup$
It is not accurate to refer to packed arrays asPackedArray. There is no such head. A packed array is just an alternative storage format forLists with machine-representable numeric elements, not an expression type.
$endgroup$
– Szabolcs
Apr 18 at 10:18
add a comment |
$begingroup$
I just tried to understand the differences between NumericArray and PackedArray in MMA12. The memory footprint of a NumericArray appears to be less than for a PackedArray. However, what is the use of the NumericArray, if simple matrix multiplication of NumericArrays
A.B
returns unevaluated? In fact, Flatten[] works with NumericArray[] but Dot[] does not.
So, can someone explain to me the purpose and/or use of NumericArray[]?
data-types
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
$endgroup$
I just tried to understand the differences between NumericArray and PackedArray in MMA12. The memory footprint of a NumericArray appears to be less than for a PackedArray. However, what is the use of the NumericArray, if simple matrix multiplication of NumericArrays
A.B
returns unevaluated? In fact, Flatten[] works with NumericArray[] but Dot[] does not.
So, can someone explain to me the purpose and/or use of NumericArray[]?
data-types
data-types
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
asked Apr 18 at 9:18
Michael WeyrauchMichael Weyrauch
26527
26527
$begingroup$
Some details here: chat.stackexchange.com/transcript/message/49966830#49966830
$endgroup$
– Carl Lange
Apr 18 at 9:23
2
$begingroup$
It is not accurate to refer to packed arrays asPackedArray. There is no such head. A packed array is just an alternative storage format forLists with machine-representable numeric elements, not an expression type.
$endgroup$
– Szabolcs
Apr 18 at 10:18
add a comment |
$begingroup$
Some details here: chat.stackexchange.com/transcript/message/49966830#49966830
$endgroup$
– Carl Lange
Apr 18 at 9:23
2
$begingroup$
It is not accurate to refer to packed arrays asPackedArray. There is no such head. A packed array is just an alternative storage format forLists with machine-representable numeric elements, not an expression type.
$endgroup$
– Szabolcs
Apr 18 at 10:18
$begingroup$
Some details here: chat.stackexchange.com/transcript/message/49966830#49966830
$endgroup$
– Carl Lange
Apr 18 at 9:23
$begingroup$
Some details here: chat.stackexchange.com/transcript/message/49966830#49966830
$endgroup$
– Carl Lange
Apr 18 at 9:23
2
2
$begingroup$
It is not accurate to refer to packed arrays as
PackedArray. There is no such head. A packed array is just an alternative storage format for Lists with machine-representable numeric elements, not an expression type.$endgroup$
– Szabolcs
Apr 18 at 10:18
$begingroup$
It is not accurate to refer to packed arrays as
PackedArray. There is no such head. A packed array is just an alternative storage format for Lists with machine-representable numeric elements, not an expression type.$endgroup$
– Szabolcs
Apr 18 at 10:18
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
NumericArrays are unrelated to packed arrays.
"Packed array" refers to an internal efficient storage format used for Lists. There is no such head as PackedArray. Normally, packed arrays are not observable. Their usage is completely transparent. The only way to tell that a List is packed is either by performance measurements or by using some special developer functions (such as PackedArrayQ).
NumericArrays are an expression type in their own right.
NumericArray basically just provides a space-efficient and exact storage format. On a typical modern system, Mathematica stores both machine integers and machine reals in 64 bits. Even if you use BinaryRead to read a file byte-by-byte, 64-bits will be wasted for storing each byte. Clearly, 8 bits would be enough. NumericArray enables that, not only for bytes, but for all typical integer and floating point formats.
NumericArray is not really useful for computation at this point. It is useful as an exchange format. It can wrap raw arrays in C, so it is useful with LibraryLink. There's a LibraryLink API for it in M12.0. It can be serialized to WXF, which is partially used by ExternalEvaluate to communicate with Python. It appears in the InputForm of Image and Audio, thus serializing these atomic expressions into a compound one will not waste any space.
If you want to do computations, you'd do one of the following:
Extract the values from the
NumericArrayusingNormal(and thus increase the memory usage)Put it in a computable format like
ImageorAudioAs a special case, neural net functions handle numeric arrays efficiently. You can give one as input to e.g. a linear layer and get another one as output.
Write your own functions in C to handle them (LibraryLink). It is a convenience for C programmers to use NumericArray because now they can choose their own data format and have Mathematica adapt to it, rather than having to adapt to Mathematica's format.
Perhaps in the future more functions will handle numeric arrays without needing to extract data from them.
NumericArray did exist in previous versions in the form of the undocumented RawArray. I believe these are the same thing as the LibraryLink RawArray API works with NumericArrays and vice versa. Also, old RawArrays now always appear as NumericArray in M12.0. The syntax of NumericArray is slightly different from the old RawArray though.
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "387"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f195491%2fnumericarray-versus-packedarray-in-mma12%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
NumericArrays are unrelated to packed arrays.
"Packed array" refers to an internal efficient storage format used for Lists. There is no such head as PackedArray. Normally, packed arrays are not observable. Their usage is completely transparent. The only way to tell that a List is packed is either by performance measurements or by using some special developer functions (such as PackedArrayQ).
NumericArrays are an expression type in their own right.
NumericArray basically just provides a space-efficient and exact storage format. On a typical modern system, Mathematica stores both machine integers and machine reals in 64 bits. Even if you use BinaryRead to read a file byte-by-byte, 64-bits will be wasted for storing each byte. Clearly, 8 bits would be enough. NumericArray enables that, not only for bytes, but for all typical integer and floating point formats.
NumericArray is not really useful for computation at this point. It is useful as an exchange format. It can wrap raw arrays in C, so it is useful with LibraryLink. There's a LibraryLink API for it in M12.0. It can be serialized to WXF, which is partially used by ExternalEvaluate to communicate with Python. It appears in the InputForm of Image and Audio, thus serializing these atomic expressions into a compound one will not waste any space.
If you want to do computations, you'd do one of the following:
Extract the values from the
NumericArrayusingNormal(and thus increase the memory usage)Put it in a computable format like
ImageorAudioAs a special case, neural net functions handle numeric arrays efficiently. You can give one as input to e.g. a linear layer and get another one as output.
Write your own functions in C to handle them (LibraryLink). It is a convenience for C programmers to use NumericArray because now they can choose their own data format and have Mathematica adapt to it, rather than having to adapt to Mathematica's format.
Perhaps in the future more functions will handle numeric arrays without needing to extract data from them.
NumericArray did exist in previous versions in the form of the undocumented RawArray. I believe these are the same thing as the LibraryLink RawArray API works with NumericArrays and vice versa. Also, old RawArrays now always appear as NumericArray in M12.0. The syntax of NumericArray is slightly different from the old RawArray though.
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
$endgroup$
add a comment |
$begingroup$
NumericArrays are unrelated to packed arrays.
"Packed array" refers to an internal efficient storage format used for Lists. There is no such head as PackedArray. Normally, packed arrays are not observable. Their usage is completely transparent. The only way to tell that a List is packed is either by performance measurements or by using some special developer functions (such as PackedArrayQ).
NumericArrays are an expression type in their own right.
NumericArray basically just provides a space-efficient and exact storage format. On a typical modern system, Mathematica stores both machine integers and machine reals in 64 bits. Even if you use BinaryRead to read a file byte-by-byte, 64-bits will be wasted for storing each byte. Clearly, 8 bits would be enough. NumericArray enables that, not only for bytes, but for all typical integer and floating point formats.
NumericArray is not really useful for computation at this point. It is useful as an exchange format. It can wrap raw arrays in C, so it is useful with LibraryLink. There's a LibraryLink API for it in M12.0. It can be serialized to WXF, which is partially used by ExternalEvaluate to communicate with Python. It appears in the InputForm of Image and Audio, thus serializing these atomic expressions into a compound one will not waste any space.
If you want to do computations, you'd do one of the following:
Extract the values from the
NumericArrayusingNormal(and thus increase the memory usage)Put it in a computable format like
ImageorAudioAs a special case, neural net functions handle numeric arrays efficiently. You can give one as input to e.g. a linear layer and get another one as output.
Write your own functions in C to handle them (LibraryLink). It is a convenience for C programmers to use NumericArray because now they can choose their own data format and have Mathematica adapt to it, rather than having to adapt to Mathematica's format.
Perhaps in the future more functions will handle numeric arrays without needing to extract data from them.
NumericArray did exist in previous versions in the form of the undocumented RawArray. I believe these are the same thing as the LibraryLink RawArray API works with NumericArrays and vice versa. Also, old RawArrays now always appear as NumericArray in M12.0. The syntax of NumericArray is slightly different from the old RawArray though.
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
$endgroup$
add a comment |
$begingroup$
NumericArrays are unrelated to packed arrays.
"Packed array" refers to an internal efficient storage format used for Lists. There is no such head as PackedArray. Normally, packed arrays are not observable. Their usage is completely transparent. The only way to tell that a List is packed is either by performance measurements or by using some special developer functions (such as PackedArrayQ).
NumericArrays are an expression type in their own right.
NumericArray basically just provides a space-efficient and exact storage format. On a typical modern system, Mathematica stores both machine integers and machine reals in 64 bits. Even if you use BinaryRead to read a file byte-by-byte, 64-bits will be wasted for storing each byte. Clearly, 8 bits would be enough. NumericArray enables that, not only for bytes, but for all typical integer and floating point formats.
NumericArray is not really useful for computation at this point. It is useful as an exchange format. It can wrap raw arrays in C, so it is useful with LibraryLink. There's a LibraryLink API for it in M12.0. It can be serialized to WXF, which is partially used by ExternalEvaluate to communicate with Python. It appears in the InputForm of Image and Audio, thus serializing these atomic expressions into a compound one will not waste any space.
If you want to do computations, you'd do one of the following:
Extract the values from the
NumericArrayusingNormal(and thus increase the memory usage)Put it in a computable format like
ImageorAudioAs a special case, neural net functions handle numeric arrays efficiently. You can give one as input to e.g. a linear layer and get another one as output.
Write your own functions in C to handle them (LibraryLink). It is a convenience for C programmers to use NumericArray because now they can choose their own data format and have Mathematica adapt to it, rather than having to adapt to Mathematica's format.
Perhaps in the future more functions will handle numeric arrays without needing to extract data from them.
NumericArray did exist in previous versions in the form of the undocumented RawArray. I believe these are the same thing as the LibraryLink RawArray API works with NumericArrays and vice versa. Also, old RawArrays now always appear as NumericArray in M12.0. The syntax of NumericArray is slightly different from the old RawArray though.
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
$endgroup$
NumericArrays are unrelated to packed arrays.
"Packed array" refers to an internal efficient storage format used for Lists. There is no such head as PackedArray. Normally, packed arrays are not observable. Their usage is completely transparent. The only way to tell that a List is packed is either by performance measurements or by using some special developer functions (such as PackedArrayQ).
NumericArrays are an expression type in their own right.
NumericArray basically just provides a space-efficient and exact storage format. On a typical modern system, Mathematica stores both machine integers and machine reals in 64 bits. Even if you use BinaryRead to read a file byte-by-byte, 64-bits will be wasted for storing each byte. Clearly, 8 bits would be enough. NumericArray enables that, not only for bytes, but for all typical integer and floating point formats.
NumericArray is not really useful for computation at this point. It is useful as an exchange format. It can wrap raw arrays in C, so it is useful with LibraryLink. There's a LibraryLink API for it in M12.0. It can be serialized to WXF, which is partially used by ExternalEvaluate to communicate with Python. It appears in the InputForm of Image and Audio, thus serializing these atomic expressions into a compound one will not waste any space.
If you want to do computations, you'd do one of the following:
Extract the values from the
NumericArrayusingNormal(and thus increase the memory usage)Put it in a computable format like
ImageorAudioAs a special case, neural net functions handle numeric arrays efficiently. You can give one as input to e.g. a linear layer and get another one as output.
Write your own functions in C to handle them (LibraryLink). It is a convenience for C programmers to use NumericArray because now they can choose their own data format and have Mathematica adapt to it, rather than having to adapt to Mathematica's format.
Perhaps in the future more functions will handle numeric arrays without needing to extract data from them.
NumericArray did exist in previous versions in the form of the undocumented RawArray. I believe these are the same thing as the LibraryLink RawArray API works with NumericArrays and vice versa. Also, old RawArrays now always appear as NumericArray in M12.0. The syntax of NumericArray is slightly different from the old RawArray though.
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
edited Apr 18 at 10:50
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
answered Apr 18 at 10:13
SzabolcsSzabolcs
165k15451954
165k15451954
add a comment |
add a comment |
Thanks for contributing an answer to Mathematica Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f195491%2fnumericarray-versus-packedarray-in-mma12%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Some details here: chat.stackexchange.com/transcript/message/49966830#49966830
$endgroup$
– Carl Lange
Apr 18 at 9:23
2
$begingroup$
It is not accurate to refer to packed arrays as
PackedArray. There is no such head. A packed array is just an alternative storage format forLists with machine-representable numeric elements, not an expression type.$endgroup$
– Szabolcs
Apr 18 at 10:18