Conversation
* allow gemma3, gemma3_text and gemma3n
* add l2Normalized method * move arange from Bert
* simplify L2Normalisation * filter unused lm_head keys
| /// - Returns: An `MLXArray` with the same shape as the original, normalized along `axis`. | ||
| /// | ||
| /// - Complexity: O(n), where n is the total number of elements in the array. | ||
| public func l2Normalized(axis: Int = -1, eps: Float = 1e-12) -> MLXArray { |
There was a problem hiding this comment.
Where did this come from? I wonder if this should be in mlx-swift or even in mlx?
There was a problem hiding this comment.
I think the core idea was to create a proxy Normalizer Scope to generalize and make this more obvious L2Normalisation (and other layers maybe) which should always inlined or inlinable, but it shifts more and more to this extension approach.
I take a look into Swift-MLX there are Linalg.norm() which could solve this.
There was a problem hiding this comment.
@davidkoski this method depends on the Linalg.norm() method, so moving this to MLX comes with a lot of code movement, but I could make an pull request for Swift- MLX and refactor this occurrence after its merged.
|
|
||
| import Foundation | ||
| import MLX | ||
| @preconcurrency import MLX |
There was a problem hiding this comment.
Why is this needed? It is either a sign of a bug in mlx-swift or in this test code.
There was a problem hiding this comment.
This happened as I wrote the test. I had imported a deprecated method and the linter judged me, but it is not longer used.
2 participants
Proposed changes
Please include a description of the problem or feature this PR is addressing. If there is a corresponding issue, include the issue #.
Checklist
Put an
xin the boxes that apply.pre-commit run --all-filesto format my code / installed pre-commit prior to committing changes